{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 sklearn中的回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn中的线性模型基本上都放在linear_model这个模块中，这个模块下包含了很多种类的回归算法，比如经典线性回归、带变量选择的线性回归、贝叶斯回归、异常值鲁棒性回归、广义线性回归等，当然linear_model模块下的回归算法，还有其他算法的回归器也可以进行回归预测，比如：KNN回归器、决策树回归器、随机森林回归器以及XGBoost回归器（非sklearn原装算法）。本次课程中，我们主要讲解普通线性回归、多项式回归、岭回归、LASSO以及弹性网等回归模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 线性回归的基本原理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在机器学习中，线性回归是最简单的回归算法。当我们只用一个x来预测y，就是一元线性回归，也就是在找一个直线来拟合数据。表达式为：$$y=\\beta_0+\\beta_1x$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个表达式其实就是我们的**预测函数**。当我们知道了$\\beta_0、\\beta_1$的值，带入已知的$X$我们就可以预测出我们的目标值了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下图是一组数据画出来的散点图，横坐标代表广告投入金额，纵坐标代表销售量，**线性回归就是要找一条直线，并且让这条直线尽可能地拟合图中的数据点**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/image-20220114075304578.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们得到的拟合方程是$y = 0.0512x + 7.1884$，此时当我们获得一个新的广告投入金额后，我们就可以用这个方程预测出大概的销售量。当代入x=0就能得到唯一的$\\hat{y}$ ，$\\hat{y}=7.1884$，这里的$\\hat{y}$表示的是一个估计值，而不是真实观测到的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 如何判断哪一条拟合直线是最优的？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "那既然是用直线拟合散点，为什么最终得到的直线是y = 0.0512x + 7.1884，而不是下图中的y = 0.0624x + 5呢？这两条线看起来都可以拟合这些数据啊……毕竟数据不是真的落在一条直线上，而是分布在直线周围，所以我们要**找到一个评判标准来评价哪条直线才是最“合适”的**。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/image-20220114080610250.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们需要引入一个概念：**残差**。简单来说，**残差指的就是真实值和预测值之间的差值**。表达式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$e=y-\\hat{y}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/image-20220114082644933.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于某个广告投入$x_i$，我们有对应的实际销售量$y_i$，和预测出来的销售量$\\hat{y_i}$（通过将$x_i$代入公式$y=\\beta_0+\\beta_1x$计算得到），计算 $e_i=y_i-\\hat{y_i}$ 的值，再将其平方（为了消除负号），对于我们数据中的每个点如此计算一遍，再将所有的${e_i}^2$相加，就能量化出拟合的直线和实际之间的误差。\n",
    "\n",
    "用公式表示就是："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 = \\sum_{i=1}^{n}(y_i - (\\hat{\\beta}_0+\\hat{\\beta}_1x_i))^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个公式是**残差平方和**，即SSE（Sum of Squares for Error）。，在机器学习中它是回归问题中最常用的损失函数。**损失函数是衡量回归模型误差的函数，也就是我们要的“直线”的评价标准。这个函数的值越小，说明直线越能拟合我们的数据**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 残差平方和SSE（Sum of Squares for Error），计算公式为：$SSE=\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2$ <br>\n",
    "> 回归平方和SSR（Sum of Squares for Regression），计算公式为：$SSR=\\sum_{i=1}^{n}(\\hat{y}_i-\\bar{y})^2$<br>\n",
    "> 总离差平方和SST（Sum of Squares for Total），计算公式为：$SST=\\sum_{i=1}^{n}({y}_i-\\bar{y})^2$<br>\n",
    "<!-- > 三者之间的关系为：$SST=SSE+SSR$<br><br> -->\n",
    "> <center><img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/fig-ch12_04_05m.jpg\" style=\"zoom:60%;\"><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 如何计算预测函数的系数？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "那么，预测函数 $y=\\beta_0+\\beta_1x$ 中的系数 $\\beta_0$ 和 $\\beta_1$ 具体的值究竟是如何计算出来的呢？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们知道，两点确定一线，有两组x，y的值，就能算出来 $\\beta_0$ 和 $\\beta_1$ 。但是现在我们有很多点，且并不正好落在一条直线上，这么多点每两点都能确定一条直线，这到底要怎么确定选哪条直线呢？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还记得前面刚刚讲过，损失函数实际上指的就是真实值和预测值之间的差异，因此我们可以**用损失函数来衡量直线“好坏”的标准，损失函数值越小，说明直线越能拟合数据。**所以我们现在的目标就可以转化成："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\min_{\\beta_0 , \\beta_1}\\sum_{i=1}^{n}(y_i - (\\hat{\\beta}_0+\\hat{\\beta}_1x_i))^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面公式中 $\\beta_0$ 和 $\\beta_1$  未知，有两个未知参数的二次方程，如果想要求整体的最小值，分别对两个参数 $\\beta_0$ 和 $\\beta_1$ 求偏导并令其等于0，将已知的 $x_i$ 和 $y_i$ 带入偏导方程就可以直接算出 $\\beta_0$ 和 $\\beta_1$ 了。这个方法就叫做**最小二乘法**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 在统计学中，使用最小二乘法来求解线性回归的方法是一种”无偏估计“的方法，这种无偏估计要求因变量，也就是标签的分布必须服从正态分布。这是说，我们的y必须经由正态化处理（比如说取对数，或者使用QuantileTransformer或者PowerTransformer这些功能）。在机器学习中，我们会先考虑模型的效果，如果模型效果不好，那我们可能考虑改变因变量的分布。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上用一元线性回归解释了线性回归的基本原理，实际上多元线性回归也是如此，只不过是特征的个数由一变成多而已，本质上是其实并没有变化。对于一个有$n$个特征的样本$i$而言，它的回归结果可以写作："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\hat{y}_i= w_0 + w_1x_{i1} + w_2x_{i2}+...+w_nx_{in}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$w$被统称为模型的参数，其中$w_0$被称为截距(intercept)，$w_1$\\~$w_n$被称为回归系数(regression coefficient)，也就是上面的$\\beta$。这个表达式中$y$是我们的目标变量，也就是标签。$x_{i1}$~$x_{in}$是样本$i$上的不同特征。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在实际工作中，我们的原始数据基本上都是很多个特征，所以之后我们都以多元线性回归为例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4 使用sklearn实现多元线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在sklearn中，多元线性回归算法由以下类实现：<br>\n",
    "*class* `sklearn.linear_model.LinearRegression`(*fit_intercept=True*, *normalize='deprecated'*, *copy_X=True*, *n_jobs=None*, *positive=False*)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 参数              | 含义                                                         |\n",
    "| ----------------- | ------------------------------------------------------------ |\n",
    "| **fit_intercept** | 布尔值，可不填，默认为True<br>是否计算此模型的截距。如果设置为False，则不会计算截距 |\n",
    "| **normalize**     | 在1.2版本中将被删除|\n",
    "| **copy_X**        | 布尔值，可不填，默认为True<br>如果为真，将在X.copy()上进行操作，否则的话原本的特征矩阵X可能被线性回归影响并覆盖 |\n",
    "| **n_jobs**        | 整数或者None，可不填，默认为None<br>用于计算的线程数。只在多标签的回归和数据量足够大的时候才生效。除非None在joblib.parallel_backend上下文中，否则None统一表示为1。如果输入 -1，则表示使用全部的线程来进行计算。 |\n",
    "|**positive**|布尔值，默认为False，设置为True时，强制系数为正值。仅密集数组支持此选项。<font color=\"red\">（0.24版本以上才有此参数<font>）|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性回归是机器学习中最简单的回归类算法，从其参数就可以看出，仅仅有四个参数就可以完成一个完整的算法。并且看得出，这些参数中并没有一个是必填的，更没有对我们的模型有不可替代作用的参数。这说明，线性回归的性能，往往取决于数据本身，而并非是我们的调参能力，线性回归也因此对数据有着很高的要求。幸运的是，现实中大部分连续型变量之间，都存在着或多或少的线性联系。所以线性回归虽然简单，却很强大。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们尝试使用sklearn来完成线性回归建模。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:36:40.089912Z",
     "start_time": "2024-03-04T07:36:40.078259Z"
    }
   },
   "outputs": [],
   "source": [
    "# 导入模块和包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:30:10.393355Z",
     "start_time": "2024-03-04T07:30:10.354803Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.33</td>\n",
       "      <td>-0.77</td>\n",
       "      <td>4.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.32</td>\n",
       "      <td>-0.99</td>\n",
       "      <td>1.38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.07</td>\n",
       "      <td>-1.44</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.56</td>\n",
       "      <td>0.30</td>\n",
       "      <td>1.82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.63</td>\n",
       "      <td>0.22</td>\n",
       "      <td>-2.48</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X1    X2     y\n",
       "0  1.33 -0.77  4.44\n",
       "1 -0.32 -0.99  1.38\n",
       "2 -1.07 -1.44  0.30\n",
       "3  0.56  0.30  1.82\n",
       "4 -1.63  0.22 -2.48"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入数据集\n",
    "df = pd.read_csv(\"LinearData.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:37:11.951636Z",
     "start_time": "2024-03-04T07:37:11.940284Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 3)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABIZklEQVR4nO2df5hcZZXnv6eqqzvVaUynk+wy6SRD3GVwhxCMaZXdZHAhCDJCCFFbRdRZxIyDDgiaEMZZEhhnyY8dYNgZF0NgfVAYCRJihGEh/FAn7IAkhgQYRWZBSHfjml8dSbrSVV119o9bt+rWrfe9v2/Vre7zeZ48SVffeu9blapz3/t9v+ccYmYIgiAIrUuq2RMQBEEQwiGBXBAEocWRQC4IgtDiSCAXBEFocSSQC4IgtDhtzTjp9OnT+ZRTTmnGqQVBEFqW3bt3H2TmGfbHmxLITznlFOzatasZpxYEQWhZiOhN1eMirQiCILQ4EsgFQRBaHAnkgiAILY4EckEQhBZHArkgCEKL0xTXSiLYtwV46mbg6AAwZRaw5EZgfn+zZyUIiWXbnkFsfPxVDA3nMLM7i5UXnIZlC3qbPS0BEQVyIroWwJUAGMBLAP4LM5+IYuxY2LcF+NHVQCFn/Hx0v/EzIMFcEBRs2zOIG7a+hFyhCAAYHM7hhq0vAYAE8wQQWlohol4AVwPoY+Z5ANIAPhV23Fh56uZqEDcp5IzHBUGoY+Pjr1aCuEmuUMTGx19t0owEK1Fp5G0AskTUBqATwFBE48bD0QF/jwvCBGdoOOfrcaGxhA7kzDwI4L8DeAvA2wCOMvMT9uOIaAUR7SKiXQcOHAh72nBMmeXvcUGY4Mzszvp6XGgsUUgrUwFcAmAugJkAJhPR5fbjmHkTM/cxc9+MGXWlAhrLkhuBjO0DmMkajwuCUMfKC05DNpOueSybSWPlBac1aUaClSg2O88D8AYzHwAAItoK4D8B+F4EY8eDuaE5UVwr4tARQmJuaEbtWhEnTDREEcjfAnAWEXUCyAFYAiD5FbHm90+MYCYOHSEili3ojTTIihMmOqLQyJ8H8AMAP4dhPUwB2BR2XCEixKEjJBRxwkRHJD5yZl4DYE0UYwkRIw4dIaGIEyY6JEV/vCMOHSGhiBMmOiSQj3fGg0Nn3xbgtnnA2m7j731bmj0jIQKicMJs2zOIReuextzVj2LRuqexbc9g1NNsCSZurZWJgt2hk51q/Lx1hfFY0h0sslnb8uicKWGdMLJZWoWYueEn7evrY2n11gTsQREwVucX35HcoHjbPCN425kyG7j25cbPR/CFPdgCxqr7luVnhA62i9Y9jUGFnt7bncWzq88NNXZSIaLdzNxnf1yklYlEKzpYZLO2pYnTmSKbpVUkkE8kWjEoymZtSxNnsJXN0ioSyCcSTkExqRuK42GzdoJh3YBMESmPiSLYStmAKrLZGQWtkgK/5Ea1Rn7q+cndUJxo5RRipBHp8HZNvKjYg4sq2MZVNqAVkUAelqS5KpwuKrqg6KSde30NcV7MJko5hRgJ6/DwehFQaeIAQASYMX1SJjohIOqyAa2KBPKwRBEEo8LLRUUVFLeuUI/nVTtP2sVMqMNp09EtEPq5COi0b+vC/MhIYcLaBONCNPKwJGkDMagrJeyGYiu6YSYYYTYd/ThPvGrfUlMlWiSQmwTd7EuSqyLoRSXMhuK+LWqft5fzCoHxm9EYxuHh5yKw8oLTkEmrNzi9jiv4RwI5UJUGju4HwFVpwEswT5KrIuhFZX6/kRQ0ZTYAMv72kiRkvm9+5yOEwpQ6BodzYFSlDqdgHsbh4fsi4DHHcCLaBONCAjkQThoIGgTjIMxFZX6/kSm5dtj428v8Ve+b3/MmjaTaMC0ESbJZtqAXtyw/A73dWRCM7Eev2ZV+LgIbH38VhVJ9JLev0aWmSrTIZicQXudOiqui0VY9p/cnyWn/Olpk0zao3h3U4eHH5qfd7IRx8ZCaKvEggRwwAp6ynkcLSgONvKho37fZiQp8nkmSA8mBmd1ZZY2ROKUKrxcB3dzC1D8J47iZKIi0AiRL524lVO9buh3IH0+0NKElSQ4kB5KS0aiSO+KYm9RUcUcCOZAsnbuVsL9v2R7DMJw7DN+bxkkgSQ4kB8Lo3VGh23AFEPncpKaKO5GUsSWibgCbAcyDIYddwcz/rDteytiOA1SZnE/d3NolZ1uxzG8DUGV1bnz81YaVkI2zFG6roStjG5VG/rcA/jczf5yI2gF0RjTu+KdV6rRY0W0K6hwsCZMmtLRgXZe466foNhpVafhAPHKH1FRxJ3QgJ6J3ATgbwJ8AADPnAeTDjjshaBGXRB26TUFKA6z4gidMmnAkKQ4kDzTCzaHbaEwTKQtixSV3SE0VZ6JYkb8bwAEA/4uIzgSwG8A1zHzcehARrQCwAgDmzJkTwWkjoNmr4bAuiWbNX7fC5qIhRdilCdk0rhDlCloXZNdufyWyc+hW2EVmZDPpOrljIpaQTQJRbHa2AXgfgP/JzAsAHAew2n4QM29i5j5m7psxY0YEpw1JmGzOqAjjkoh6/n4SYbSbgrOTs2mcwMSeIBmZTuiC7HCu4HgOP8k1uhW2uYnptqkpiTyNIYoV+QCAAWZ+vvzzD6AI5IkjCZ7hMP71SEvP7oeRe1e+VXaTeHR1zc07gmZLEwmVrKL2Q+s823asWZ9rt7+C4Vyh8js3OWblBacpNxrtDZRVSCJP4wi9Imfm3wDYT0TmPdUSAP8SdtzYSYJnOIx/Pez8a1b0QF2BDKcSBVHaNeNYOSe0GmPUfmiVZ1uHGUStQdzELb3fWj+8O5vx7BaJs1+nUEtUrpU/B3Bf2bHyOoD/EtG48ZGEbM4wLomw83eqk2LidFGIYuUd18o5CRdpBVFnZKrcHCP5MRwZqQ/WaSKt0wRQX0xUtr/RsZLn+UkiT+OIJJAz84sA6ryNicZJHmgkQQNi2Pl7CWpxX9TcVs5BN3KTcJFW4CRTBMUub+g8105BHFBfTHQr6pt+9EqodH1J5ImeiZvZ2erZnGHn7xbU4r6oOdYx3x9uIzehJRcakZGpO0evQ/A0Lyb2jUmd/n5kpOBp0zIppQQmApFkdvpFMjsTgCqL0dzwnDLb+wo4iAVSeW7rNHR+dB8Zos22liaMbXsGsfLBvXUlZqd2ZrDm4tMBoG4lb9n+rsNrBmcjGj5PJOLO7BRajSiyGN00bl0wdatjHkWGaBLcM0lD0bjHXMepZBSnJZ5XnVsSeRqDrMiF4Nw2T19bRaXhAwClAHbYMFt+V+vXbEkgTlJJigBFLwhH4qipIrijW5FPXI1cCI+TO0S36nYK4tkem6/dQgI07lbGaQXtFMS7sxnRuVsACeRCcJzKvvq1+qUyQP6YzddeDubWjdwEZmy2AkGcIpk0Ye3S05teMldwRzTyRtPMTbhHrgN2f8fYSKQ0sPBPgItuDT6ekwVSJ4+omDLbaEaRO2z7BdfKKQnN2HQj7IbfX257Cf/w/H4UmZEmwqc/OBvfXHaGrzmorI+ulFfqonMnHwnkjaSZgeiR64Bdd1d/5mL1Z7/B3Hoxyk4F2rJA7kj9hcnJmWJC6fLqXXN/b13Z+ylLkBDXStg09b/c9hK+99xblZ+LzJWfzWDudqEwf+8riAMolBhf27LX81yF5iHSSiNpZur47u/4e1yHvVhX7jAwlgOWbzJWzmawNH3u2R7n8bgIR3+EVb7xmrGZhIJoZcKmqf/D8+q7GvNxt0Jc1t8HocgcqrCX0BgkkDeSZqaOq3zZTo/r8HMxmt8PXP+G4USZMtt4jNK1fzth3+D02ootQbVWwqapq2p+Wx93u1AEWYnbkfooyUcCeSNpZk9IXeD0ElCtaC9G+4GbeoC1U+o3Ief3G6v1tUeBNYeNv53cK7pMVVXGJsg4t/WcTnPUbZRWNlGn6F9HAML2m0yTwvxd5pTVj2pX2uaFwumC0dudxeVnzXHM+jQZHM7JqjzBiEbeSJw2B5003aDZk9bnnLIYeOMn9ce1TTKODVvHBKiu7r1o/9p6KA5e8ZokJofSu9o5UvVx6/FA7f+Ln9ehwKpZT8lmkE4RihaPXyZNlZR4q7Z9zntm4JlfHqjRuj/9wdk1GrlXzAuFU6nbweEcHto9iFuWG1r7yh/sRaGol7lW/kD08qQiCUGNRhWUAX3TX6ff6YKLronwrA8Av/6n+tWw1wbD+7YAj12vcJdoyPYA7ZP1Fye316W7gO3bAjz8JX0avzIZSZNwbko+Tg4b1cVFMzdV0So7mRThkx+YjYd2Dzoel82k8bGFvXho9wByBe9VBzMpQtekNgyPFNDdmcHRkQKcnm2uyL3o6Gki/E3/mRLMm4QuIUgCeRJwypAE/K9coxjPHqhOPR/Ye7+7C8UJr4Ha/J0q0J95mcs8CFg7XD+2NlCb0oXT96A8ponDRWjRP073HBB1+ndYMmlyXFnb8fIOWJmoHeyTgATyJLO2G+qvkdNXzBZcohzPqaBWWJxW6VZ0FyM3dBe4IBc33ZgOY839f+ujeJcCEzTdHvC2Irc+R1L0G4+k6DeKIJmHTpugut9RSj+203jZqfp5mOMp0+sjCk+5w/W2wEeuq3/Pgjh5nNL4nUrbKjdRy9NFB645cHFtv0nN3PjoAFIOm5ONwG8QN9Pt/XQbAqQ5RNKQQB4lQf3LQYIMF/VjB6rHzdXxAtshAwSxQg7YdU/te7Z1BXxfOChlJCZtXaG+gDrVb6/5HQBKG55sno7r81/AD0uLa/3ZmgvlEE+LTS6JA2u6vb2O+dTODLIZfXiQ5hDJonWklYRk6jnidPvuVrXPTS922txTja3bVN36Red5eJEatEQkv/hklNNIUwlt1nOnMsCybwX+jOiqBfZ2Z/HsHx+sk55KAIiN4L9hrB/bS4sBGO9IZ3sax/PG/113NqPsm9lInDRulZPGvikrGnnziF0jJ6I0gF0ABpn5IqdjfQdyLw6HJOCkTds3y/xelLyObcev0wRkZGlGrpET0N5p1FSJkDFO4QQy6KLR+l9me4yEpADMXf2o9tXe/sn3Yln6WbUNEsAIt2N14cpKMLfSnEtdlV6HWi+6NnEfW9hbZ4uUIN4cGqGRXwPgFxGOVyVBmXqOGriXhJ+g8kuQZKJ9W4BtV/kI4uXxVDJE3xVaHdkVShvPT3eofhlsTBh68HWFL2EyFEEccHzd9rZm9mQXJ+nghq0vYVtxkXEnNGU27KG5k/JY1ab+/2xmECcAz64+VxuEdVmiz/zyAJ5dfS7eWPdRx+cLzSOSQE5EswB8FMDmKMarIyld0d2CsBdtOuhFya/ubcoxJT+38VQdb36/8W/Turf7O8Y8/aTYA4bEkZlkFOiyB9ZsT+ALRImB7xbPU656a1BcIHX1SV7Y/u3KRXoHXYWPt/8f5ZDWlHXWfAZn0iFfrycKlqZ2Ymf71Xi94zLsbL8aS1M7a36fInLMzgxSTsDtgig0hqgyO28HsArASRGNV0tSuqK7Vd/z0j7NT+En+zgX3+FNkjEvOH7rqJjrxYrWbxECzLG46NHPDSAz2biQ6OSUE0eBOWcZf9y0+1QG6DipUmXx5uMfw3dGPwAAOIIu9OCY+nmKyoiqleeHiz/BvJ/fDZRX9525t/Hf0nchnyopLxZD5ZT19/M09NLBut8f4cnY2X41ZtJBDNl08zhYmtqJdZnN6KQ8AGAWHcS6zGaggMp5zQJYgJGdadfDp2j0e93dSdjKjkJ0hNbIiegiAH/MzFcR0X8G8HWVRk5EKwCsAIA5c+YsfPPNN72fJCkauZNOvXyTtyDrZUM07OsN6sHO9hiVDL0k/ZgZlLpNWK+Yr8utfvnyu6qv/ZHrUNr9v0ClEopI4TX+PbyHBqF1/q09Wvnntj2D+OoDL9YdsrP9asxK1QdkVmxgAlXv9cLf7agJoIBxt1BCCm1Uzad00s2jQDf/gdJ0LM7fUfd4dzaD4/mxmsShTJoARk2DZqeNTccNYfGYx0KcGvkiAEuJ6NcAvg/gXCL6nv0gZt7EzH3M3Ddjxgx/Z3CyjjUS3R1Adqp33dtL4aewewJBPdjmebyeY34/cOmd/s9lpZAzLgannu98nCWIY9fdSHEJREAblZyDuEUCMleQKmYqVtUAQATMShmrW1OqML3XQ8M5bC8txoPFs2v826nyvKw46eZRoJu/TuIZzhXqsj8LRUbXpDbP3YDCVnYUoiO0tMLMNwC4AQAsK/LLw45bRxK6ouuKXgHeGx54KfwUtou8U0p6tge4cL1lDpYUfGvjCS/nAIzX48sVo4CLhkzjBUX9dMccHMvdglNJ1yGejlmaYAhYAvEYsK7rYXT+8Dc4a9J0/Lf8J7Ak9SJSHvZsVUF1aWonVrVtCS3B6OY/xNN8jXNkpIA1F5/uSRrRFeQSj3njkYQgP+juDHJH1MfrAq9Z1lXheKjZULTjdU9AtzG6/C7DjmdeFK992bAuLrkR+Pm93sY2xzr1/Kp7BzA07DB4vRPwK+NkJlf+6bRS3DDWjxFudxyqlw5iffvd6My9DYBxMg5gXWazdjVsxx5UTV17VuogUoqVvx9U82cGsjjhezxVIwnVpqYqG1QaMzeHSAM5M//YzUPespi2w60rjJ+tHXGC1hnXBXpzQ9GKny7yfqWop2727m6ZMru60WntEkTk3g0oCvzWTy8cr0hcTivF7aXFWF24EodKXdBtGxWRQtZmdeykvCdL4Qi3Y8NY7fu/qm1LjbZujhdEgjHnf5ir8ycCpqWO+b442BtJ6Fw+AKQxc0KQeuRecOu16VRn3AmnmtyVBsYBM1n9SFGeNXUyLl63zatfQRfzRjGsC9e7O1CcxleFResFYuGfKCUghoMj/bHrgaduxs4TAzjSYazQu3EMQzwdP8ECnJPag5PZkDaI1FJNCUDapnubqFZDBSa8g8noxnEM8TSlZOJX13Zje2kxVvEW9KRqHTzmxWF7vvb8mRTVbGxasd69OHUhEl95MhBpxQtum49BN2OdvOFW6cPaC9ONKIt22TELdTlZKOf3I1CSTyZreMrtEk0qU9X0AaNR9NwP1T3dcVVcLtRFYPTQMfTQsYqUcVlqB2aiKm1M1dgYUwDILF9gQxX438FkvG90E75a+DMAwO2Zb9V5u4d4unK8YUxWPm5naWonnu2o9Y27XRzMjkNpMmqi67oDTclW/x9kUzP5SCD3ghfvd5DAG7Ubx2vWqD3Yn3q+N43bLNSlq6A4ZZbhKvEiNmQm17/ui2416qNYV+AdJ1XnvH6u0YJN0ekoBWglESfsMVi3cfobzMAL/+7P6y68unN247irBr5hrB95rr8pnoycqxRijt1LtWMf4S7l8aY+bxb1KjLjod2DOOc9M5BR7NQez49VdPKw7eqE+GmdolnNJEwxrEYSxqN+5mXA3u8bmrIblDK0aquubo6x6x54TkS3+LsrqOaXbgdKxUB+dUfJxSOmB3xH+kO4Z8b38cFDD7u6VMY4haPciWmp+hW+1du9u32F6zEqdL7xw9yFScjXaO9OHvbe7ixG8mM4MlK/R2L6wXU1WEQPbzxSjzwMgcrChiSIROLlzkEnE732BPCNIcPZ4rZpySXL5qZlRf3Kw/BVTUT12lTzK+bDJR0FhNkIqGYQzBWK+IODOzxZDduohB5SyzRWDXyqh2PUv1dLKN04jtWFKzFQmo4SU838VQwN5zCsCOLm7wDUlbiVTc3kIZudXvCSeh8lbpur1uOsc8pOVfu5rRq4W4f5U883sjvdMDc3zeqC+7YE8JJz/WuLoX5OieEp+NphUM2qeGlqpzY4M9fLMjqZxmpD9Or/tvvNhzVlCYZ4GraXFtdtbOqY6dAdyCqdmDXLhWQi0koSCSqRqLCn9rum7/sstDplthF8ieqbOvvBfG1Bywu4UGLjlZVQvQ11a+bDbLwTwzAsfT10TPscVSBXPW6aRMzkn4WpX+Gz6SdrLjR2KcReRwUwarATCO00pn2eG6Y8AkCkkxZBJ63IijyJBJVI7Jg2RusqXtlh3oqfCztVg27YBYF5R6Co7x0FZi/LtI+VOZExkx4ccxTamfWFu46gCyOlSeilg2BU7wxm0UFszHwbBKoJ4szArtKpNcFY5TfvoCIOlbqQ40mYSYe0Fkcdqrrk1gJaUnO8tZBAnkS8VHt0lSBILf/M7wfees5fOr6WKIOt5aIQU9XuIPKKV5iBPNrqVshrC5/D9tJi5eZkB9Xr/kTA4tQrWJraWQnKOj18Kh3HwtFNvudq1iW3ItJJayOBPIl4STByqqcCAGB1rRfA2Nh0JO4+NqrxdeeLdy46ScQPRMA0OoZRTuMwdymTgLym8QPGBWdN271YBUMTLyGFFOplK791VExM7dtaxvbzXT/DqswD6Mz9JrmtFAUt4lpJIvP7DSuftYnDmZfVSyRuBG3IMffsau/O0OY9G2bij+d0/nj3cIIGcZWS1EFFMAPvHr0Pi/N31MgcuuQfHT10rOI/b6NS3flGuB0bx9wDrf3lmbVQrGn3F6d2YlXhW5UaMp67VgmJQQI5EMzqF/d89t5f28xh7/2185rf7x4Mg9aAGfiZ4V7JZBFpIM324IUzbsKily/CwPFwHz2Gd1m+kfv5PXRMmcyjKmo1ymnt3FQOmDFOVSyFryz8Jv6R/shxLtlMGp85a47SNmhNu1dp8CjkMPLYjeG7/yTtuzVOEWnFq9Wvkbh1IjK5cL1+49LJ5+624VnIGeViI/Zuj+VH8MCutzCYn4WZHd6lBhV+BBemyO8rtBChUu7WXp52deHK8mPVzUmVa0VHGiVcU7gKP+04B+/881glS1OHznWybc9gjd1QJ/tkR97GA/xFbEj1Y/vwYv/df5L43RqnyIo8SY2dTbxKIqYEU59oDsz6gPEaVCuhmtIAGmJIwGkrnsBX8X0A/qUGFZ5kkXR7LEGcSL/S7y23WbOn5gPA4vwdNdLLmrEr8NXCVZ7uGoiAdZnNOKfwY9cg3tud1QZxe3MN3f+FvamGvSqiK0n8bo1TJJAnpbGzFT+SyGtPQLlx+MZPamuubLuqPpgvuRH6tWo8a1gzY3HDWD9G2WdJWp+UGOBiPrbVuO5CUkRKWZ52Tdu9yubI20uLMejxwtZJeXwt9YD292YhrZ0nliulDFUlQ7da7NbSur4KZSXxuzVOkUAeVEeOEz8lAbx+KUoFo5yrladuhrYHaXunt3F9YnVaUIyCB5cTgBybB/nQznXHqjYhVQ4ToHYDU1VAS9UYQoUufd9aSIs0m5aqQGzWMh/J/p76hJZz+iqUlcTv1jhFAnkz6qi44acqop8vRe5w7eaT1r7I0Ha+DwED6KQTlXRzq+c6anR1xStzcbAd2gOo07HVTUjj7yzyKGm+VvYxrCtdM5haa6QcgXMlQzu6TUurlKELxLvf9WF0Xv9Lrdw2xNP8d/9J4ndrnCIp+kB9zZJGeWijOK/XVH2TTNZDWv9koDCCuKx/eaSR4WJo/7aJ3wqHXrzj5teiBEKanN+HEgNjtmQg+zmcUvgHeTpm0kEc4S500Bgm4wQA4Bh3oIOKNeOOIY3fcSe68U5dj8/XOy7TbJqSUV4ZcK9kqPg8jXA7NmSuwns/usJ/0lCzvlvjFF2KvgTyuHD7AOvKyTrVI9eNaX985LC6HC2lvNVD8XpcE2AAN7V9FX+e34ye1LHY3Shegn6R1cHe+tXSjeFW0MvaaegIT8ZJdKImsJcY+G7xPKwZu0Jb2tZebtmaCKRMx5fgm1hiC+RENBvAvQBOhlGTaBMz/63Tc8Z9IPcSpP3WOPcT+B+5LqIU/OTBbNTcdipg1UhGuB1Z5H0X0wK8V2U0V+2ddEJZfZEZuKZwFQBgfWYzslZ5xW1xILQUcdYjHwPwNWb+DwDOAvBlIvrDCMZtXZxsV6ZGrdOndZuXfqxcuhR8v42L/RCy8bKf9cS0lHMVwqhvMlWaOTNwqNSF1YUrHZ/rVYd3G8OpDR2VU/p3v+vDeHnhN6PrOCW0DKETgpj5bQBvl//9DhH9AkAvgH8JO3bL4lTz203P1m1e+rFy6Y6N0htOaWM8a4XFm6YGlmS8rq69HBf1Sv0wd6EDBUzGaM0mahZ5LEz9KtCYTj50p+fomJY6Vi6EdS6APw00J6F1idS1QkSnAFgA4HnF71YQ0S4i2nXgwIEoT9s4vKYb64IxpZ2DuNOOvh8rl197V/tkeN4upLTRRejSO4Eps8FHB/CbrX+Ba/7iBjzP8zyXwvKK3xV21EGcGXiFfx+d5SBupZPy+Fz6ycDnNEvr+p2P8nEAv1n773HNX9wQPJ0+CJKCnwgi2+wkoi4APwHw18y81enYltTI/WjUumOdgni2x0i5d9roDHN+t6T2vi8YkoyXpg6KHprmx6jGqVF+PGj5WGbgODrQidHQJWjDVDmM87lV18ohHOHJNa4V1fPcdHVrf9HYG0ME2bAXQhFrz04iygB4CMB9bkG8ZfGjUet84E4p8e2TnT/8frzl9mOzPXBdG++6G+h5d73vV4Wih6bKt2105An+ESMCumg0tDPF76pXNY+4zn8EXbgo/T/x7tH7sDC/CfNG78Hc0ftxTeGqugShHNrxximfMi6kGkxvuu90+iBICn5iCB3IiYgA3A3gF8x8a/gpJRS/6cbz+w33ydph4+9KSrzP8U38WsLM8y/f5K0HJwD8+p+MC0CEpFFfgtUvYQKpeUfQTIeL0wq6i3I4e/QZ5e9y3F5pN3cEJ+HBsbMx+c2nwMW848a1mYU5NJwLLn14eZ6k4CeGKFbkiwB8FsC5RPRi+c8fRzBusogi3dip9KzTOOYtrLV2itd60V5awplwyZhjhO4WtwzLuInr3F4vTm7nb0exkt1psjS1Exsz3664cwhAFx/Dp9I/xsk4YNyhcBG6fQ0z8/PzXT8L9rnx+nmTFPzEEDqQM/NOZiZmns/M7y3/+ccoJpcooko3vnC9/3H83MLaV1J+GxmvnRJL5UMnGp+SFp7j6PBcG8UNe+2UNW331rWByxArShrU57QyG2UQPt7+f7Aq80Aw6cPr501S8BOD1CP3iiljhM14CzKO2y1sRXaxNS6Oo5FxDFmfCcjr8YXRnzODl0u/j/+U+gXSKKGIFE4ggy6M1h3vtkFprZ2yNLVTmfTjMBuAUmAuGUXCyGgWva5tM9pyJ9RPcZM+/JRRBiQLNAFIIPfD/P5oPqR+x3FqxvzIdcCue1AN1ioDoC2Yp9uBYkFxrAOZrFHj/I2feH+OB6LomdlozGD5R6lXKnNvQwntXMQop2tW0yPcjgeLZ+Pi9HOYivpEpjy3YUO5ZdtNbffgs0HsjOUgbqWteKLq9bfjJn14af5tEtV3QgiFVD9sBVS3sGbX+V13wz0gc63b5ZK/NzZBzce8EEMQB1oviFuxz72dxtCOYqUa4kBpOlYXrsSasSswwpOUr/UdnoTtpcVYmtrp2inIt3TDxWDSh0gmLYesyFuBmlvYAHKJrn6LW90X6/N/Xd+H0g2Vt3y8Q2SszvPchixO4PbMt7CKt2jbqU0lo7jZ6vYtnlZV5v+657f0zMvK+QE+pA+RTFoOqX7YavjdwFQlaNitjKee71xkK9tj1DIPQCtKJ3Ggex+KbDTYIOJ49gqyPcD1b0Q7ptcqnLrgL9UVA6NLCJIVeavhx6NrrYNiomqIu/d+53ECBnGgNYK4LsiWzC5DEbwG3Rhpw0sY/gQ6QvzfKdE1VH7rOeNz5NZoWRoyx4Jo5K2GV4+uqWnavxw6a1mDaMINYGCcgngjXodZZ8b5XOScMQxEW/9E9/nZ/R1vlkXJBo0FCeSthnLjU4Huy9HkrDuG0RItCCVPgc3nfMr1XII+N2qswfswd+GawlX6xsxTZlczh53KCHtNHvOC38qa9uMlGzQWJJC3GqqaKzqODtQnCGWnNmqmShiEE8j4DoKjnAaDIs8UJQLSPKacj9N54spYNcclMkrTrstsxjvoqK+JDhi1cUwuXA+kMupBo1zx+s3atB8v2aCxIIG8FbHXcdEF8+zU+lTr/DH9F74BpMBGISwfQZAZyKDo2jszKJMouv6hUdNJebyHBpUFyfDGT6sr7fn9wLJv6Qc6uj+aMrNe7wgBtWVRrI2xIIF8PKD7cgD1emQxD7Tpq+fV4LPmipcqg2Fqd7sRVOpIahA30c+Pa1fa8/td7tB81OjRYb8j1H1GKK2uzumniqfgGbEftioqC6HdL7x1BQI7IjJZw4O857tG8LfAACjbA5x+KfDaEygND2CIp+F1/rc12Y6NZELbHJffVbX/PXa9u1Ml22OUTY7C/re2G+rPGBl3jEKkxNZ8OQgSyEPiVNAfqAZ4SgUogEU1FwY+ul/pbx4oTccnO+/CygtOw8bHX8XC3+3AusxmdFJecbQ7YZOHJnQgT7cbclnheLDnqz47XgO83ybiQigkkI8ndF+ebI9RezyondD88u3bAmy7CigVtIeWmPDu0fuQzaTxvjlTsH7/5ZiVUmcvWtEFXGajycJk5Ooq/3kZE4gukCcmIzXbA5w42phqlKrPjnlX5pQZKl2CGookBI0ndFatMMkfmayxCveYOWpW7MsVinj2/x7GzA73IO6EWYjKai/0E0ijdrI0ze9uDYK3zYs+oUeH6jyFXG1BNlXyjqTzJwIJ5K2Irjqdb8o1W6bMNoK4NTPPgRFur1TsMxni6ZilqSdSc0aXgNv0VXCZpszDnokbh7fad7kF2xXNtDJaA7VUQGw64lppRfxYwBzhqpzy2hOuQZwtFf22lxbX/G7DWH9do4Uk4mel3bBVed8XgLVHqy0BTaL2Vk+ZrW9s4pRQZEeSdxKHBPJWxG7hcsLcCNNhfik9fDlLICzO31EXxAFge2kxVheuxEBpOkpMOFTqQp4bc8PndfVcZH8eHiIjCzXqbNI69t6vtgTqbKV9X6jaDMnjV9haskFl/1MFeN1nKzs1WB9QITYi+aYR0UcA/C2ANIDNzLwuinEFB6y3s2un6I+75O+Nvx/+knOTAQ9yTQqM1zsuwxBPr0grq9qMEq3mY4vz1ebNS1M7y78/hBIIbRRtZyG/BFm1pFHCEXT57Nrjk0IO2PpFQ7Kw68tt2eqdUrbHCLjz+6sNRVy7NVG9bu0khdgtrXa5LZUxkspMeUaKXiWC0K4VIkoD+BWADwMYAPACgE8z87/oniOuFURbynP9XKXuOYyT8ONLnseyBb3u7oJ9W5Db+hVkFa3KVOS5DQyu64ajkl0Ao/vN54J0v0kA9Z0xY8RqBbQ7h1KZavamlxyBKCyA9s9p/rhaYxe7YUOI07XyAQD/ysyvl0/0fQCXANAG8glP1KU8L1wP/PDLNYk7eW7DjYXPYsfWlwAAyxa4uAvm9+PlXx/BzN0bMBMH3bu/1zUCNtLJ17TdC4xVV+rD6AIz0EP1bc5ahYZOu5AzknpODNevtksF43ftk+EaxKNKe7ev3td2q48T3bypRKGR9wKw3pMPlB+rgYhWENEuItp14MCBCE7bwsRRyrO9C4xq1byvF1Zge2kxcoUiNj7+KgBgW3ERFo3egbkn7sOi0TuwrbioZoj3L/1TDC1cBQ4RuXroGDZmvo1ZqYNIkfHztJRzEI9Kg45Tx26oGzF3WC+Z5A67B8040951G7CU8q+V2wu6idYemCgCueorWve5Z+ZNzNzHzH0zZsyI4LQtTJSlPM3Vfe5wpX72JNRmVw4N57BtzyBu2PoSBodzYACDwzncsPUlbNszWHPs+//v/wj1oSCC74Qes9pfFIH4MHfFEtCPcBeKYa5wUaKtYElGur7d/QJEFzR1jiku+qvjYn5urQXdoiy3O8GIIpAPALBW6pkFYCiCcccvUZbyVKzuOymPVW3VL8TM7iw2Pv4qcoXaAGtdrVfQXEzqyqjGECytLpEglEDKTvVecDpnDh34UfEsNHer1sLoO4YbqQYC+q7Qt1aLKmiarhdVsSw/d5XSYCJSogjkLwA4lYjmElE7gE8B2B7BuOOXKEt5agLvTDoEAMhm0lh5wWkYGja+NEtTO7Gz/Wq83nEZdrZfjYW/2wEA2LZnEIvWPY2B0jTleCUAx3hS7OnrqYDhkhlIEweel/55hCITPpd+EpmYyuj6plQA2rtqLYTLNwEX3ao+PkjQdFrBz+/XSz9e7yqlwUSkhN7sZOYxIvoKgMdh2A/vYeZXQs9sPBNlWrPGNjjE09DbncXKC07DsgW9ysJWs+gg1mU2Y8s907HmjdORKxSxIdWvLH6VJqCTT7gGyrDFq4L2yIxtI5UIXTgR0+AhyB323lTZb9BUbcZvXWH05TQvFjq7qte7yrDPF2qIJCGImf+Rmf+Amf8dM/91FGOOe+zNIYJuTGlW97M+fgueXX2uYT0EsPKC07CqbUtdgO6kPBa9+a2K7GIm9qjasbnVBGefCTcqEudscfVpa54WwXvhDKmlEdVK2q+Up1rBg4Fdd1fPGfauUhpMRIpkdrY6Hgv1L1vQi5maWii/h0M1P28vLQ4kcQzydAzp+kvGQKIbOVPctkWul0ZUWvgPvwyMaGqr6LoGOckbj3y1+u82SyDO9vhzykiDiUiRolnjAY9Fi35LM3Ay6q2fb6NeF9cVwSqxemVuLaQVpi65HxK3erfQkKlZA+6+Lers3WK+rjFI7RiKHAanLN/8cSOr1J7xORagdLIU24oMWZFPIPa/byVytsJWI9yO9YX+usCjKoI1wu34bvG8cj2VqsPkMHchx+24PfMtrGrbggeLZ2OgNB3Mzu3fEr2ibgVMacSsHx+0brl949NN3tj9HfXm6cNfEvtgk5BAPkHYtmcQX/2XU3F94UoMsFHYylrJ0B5TTa38SObfgmEc+2DxbCxJvViprXJf8Vwc5i5MhZH0kyJgVuogPpH+KTaM9WPu6P34bvE8bcAuIrjV0CuJrHYYBVY9+bHrHZuAeMK6up/fD2Qm64/VXTD8esmFyJBAPk4w7YNzVz+KReuerkn0MZOBFv5uh5E6j4MY4mnYMNavrItisr20GH3HbgetHcbf4lP4RPqnlYzNWamD+Fz6SWXWptXHviT1olYCaaMSSj5ECC+Blrlc5bBccvfe8h2E292Bn3M0HbseHUXzCfvG58W36491asptlhiQjM2GIhr5OMAM1KbzxMzaBFCxHn64+BOl9RAFOAbzYjmy3TT5IXTmarVWJ43a9LHrNlhNUuA6y2KJgRJSNdUSR7i9ckfQS+paMMzAvcXzsGbsiprH1wDY2X61p1Z0SdbdK7RPjlZbVrlF5vcbdkNrhyDzWE1T7gq5w1IdscHIinwc4Ja1OTSc01oPrRmgOv5y20uYlPuNrzkxGG90XOa63ram5zMD75Q6MIwupFCq0+A/m34SgBGs7fp9iYF/Kp2OJakX8XrHZfh5xwrsbl9R+XdvBN2LEoN9I9JPUwg7lK5d3Vfsi1MMLRxcXYGbzpKLbjUSkrwiGZuxI4F8HGBmbeoe7+7MaFfG5srZie899xaGNBmfKowsy2qQ9oJ5bBeNoocMvd1ckds1+E+ln8EPih+qNLEYKE3Hd4vnoS/1Wl2xLvPfLROkPWHzkJ9+qffnWclkgUvvNP5tBu+tK6oXClML52JtYwoAyB3xN2XJ2IwVCeTjgJndmrZvZKymj44UtP5us4myGyoXi6r+Sskhs9NLlUP7c1UXgw4qYlnmOSzO34F3j96Hxfk7sCT1YkMsj76IpB2fCouHfN8WwwroZS59V9T7tgGL97w8tgpzVW2u2HXH6ToWScZmrIhGPg5YecFpWPngXhRsu3nMxmoaMAKx3d+taqKsY3tpMRYWf1XTHML825Q/Himdhc+V5Q8VUXanP4nfqXQg8iKbNAUPjawDYybz5I+rz5PtMbR0txIQt83zPk9T79Ydb+rndo+5ZGzGjqzIxwHLFvQik3bWDuw9NXVNlJ3QOVCIgG463tgOQGwkHs1KHfQl4SiG8ZhKn0Bt5uh+vWPFfHz5JucSEH4kD0rrg7hVP5eMzYYjK/JxwLY9gxgpuKfUby8txva898Btx8mBkvZYGbAEoMhtyg5DgPeiW0xAJ8JLKQQYAco1maaJvsRUGigFSPbx4hjx0KsVgLGq1q7cSdq8NRlZkbcATh5xAPU1xUNgL3O7NLWz8rso6qgQgK8XVmibNIwi7ak5hFusZwYOlboMn3q2x/kZQTMig0ApvY5cd2wa6PsCsOzO8go3AG6OEWWjCFM7s7lVdHOw6t/SMKIpSCBPOF46++hcK35ZmtpZkStMh8i6zOZKMN8w1o88h7uJIwBrM/eCNCvcDhQxwpNwb/E8ZQVG6zhOHOYu5DDJOK59MjD37KBTdsbvhqbXvnaUNiovvvaE8fO1Lzsn4jihK44FqItXLd8ErD0KrDls/G1KM14qFkrDiKZA3IRUtr6+Pt61a1fDz5t0tu0ZxMbHX8XQcA4zy7XENz7+KgYVgbq3O4tz3jMD//D8/krSjl/MzUIz5T6LE5iWOlZ33EBpOhbn76g8Z23mXkyFcVwJpJVVnGQSXfEt63MBB5kl22MUalLc7sfd/KJmDheuNzIZvWZXmqtaL3KGSSZrBNutX/Q/R9U4YfTqfVuc6+iv7YZahiLjAhFFDf4JDBHtZua+usclkCcDe3YmYHT3sSf6RIW5+ra6WHSBt8SEd4/e52schnv9crdg7ki2xwielAZz0chbafR+5JTZRjD64ZedKwxWKAczwPBr+9Hdg1wAdOPEqWffNk89R9WFN4oLywRDF8hFWkkIN/3oFWV2Zjqm6KTK9NSdyslrrnLDHOYuTwGagEodFF+k0tUVMBfBDnN3n0SIr8DRAWOF6SmIAwBbSrf6fNFHB/SNj/2OEyc6+QUQySVGJJAngG17BnFkRF29rsiMbKZWG7X/HAS3GigmeW5z9ZpvLy2uSc6ZSvXyjIpBNiSbQY+bqMzAcUyqc3AE/hBnssCl33au9OfElFn+AqN1s9BvWn12alV/NrXyIKn52anO/TjDYMou1jmaG6W6TFDJ+IyEUIGciDYS0S+JaB8RPUxE3RHNa0Lh5Drp7c7iluVnoLc7C7L8HHal7tWB8g5P8uU19zq2NRlJlTWq4jg60MkR9c+01xgJwqnne89YDJsUkz9WmzofdGU+NhqPq6TGrWKZo6mD+203J/gi7Ip8B4B5zDwfwK8A3BB+ShMPJ9eJ2Tz52dXn4o11H6304fz0BwPa0cp4DZ5T6XikY5tZoNZkJFOeOVRyth1Oxmg0Oni6HZhUrity2zyg4P81AjAcJV7kDlUbNL+1SuzyTSEXrHxtQZEJGoXE4eZWkR6dsRIqkDPzE8xsZnY8B0AurwHQ1Upxill9v98T6j/Prm3rrH5WfZwAXH7WHFx+1hztHYHphJmEfKV6ofn3QGk6rilchfeNbqpb5W8vLcZNY59DHnrZKLLtglKxHATZffOw7wv63x0dqLfvZXuqvvUps4Hldxnd7u0r/6StRMNKHLrnm49Lj85Yicy1QkQ/AvAAM39P8/sVAFYAwJw5cxa++eabkZx3PKByrFiZ2pnBmotPx7IFvZXHFq17WmlLDIrKfTLC7TUr58vPmoNvLjuj8vs//K+P1WSUOo0BoMbqqGpq4bVmuJJUJnyXHDuUsni+Fd8T0wHiZsmzs2+Ld8tiJms0OY6ieQQIyHSq70BM33pQW6DOrRK3S2aCEdh+SERPAjhZ8atvMPMPy8d8A0AfgOXs4cog9sN6z7ibJzybSeOW5WdUgvnc1Y9GnjRe9ZUf0nYQak8TNnz8TCxLP4uRx27EpJG3K4F5VdsWZSA+VOpClvKOFwkAeL3jM0h5TPWvY/ldtcHRtCfGTWaycQGxSh9OtrpHrqtv1qDD9KkDzsWq7EyZbVxUVEE73V4uU+lw0QtiCzQ1crEXxkpsPnIi+jyALwFYwswjXp4z0QN5UM94b3cWz64+F0D0K3I/LGt7Fv+94260FasbjyPcjknIK22HOn/6EKbjE5PuqlzMdtBV6My97X9CulXfTT3e0+9NB0hUwV81p31b/PnH0+3AJX9vBELrqj87VT9P63mdPN1mZURKqd+jICtpv3cmgm9i8ZET0UcAXA9gqdcgLug7+rh5r62boisvOM214mFcfD31QE0QB4xuQyWfH6eT+VBNFmvnhTf7d2OYG2YqS53XIJ7JGs0ZxiK8MKo046duhi//eDFf3Syc328E1rXDRhBWQrWbhzrdOnekOhZriq2JLbClCOta+TsAJwHYQUQvEtGdEcxp3KNzqbg1B67bFG1SQT6dBz2NkrIFm1Oi0cWpnXhg5ItYuu10jDx2o1HP2roh1vcF55/rmiNYLHU6n3W2x+LpJkMO2HV3tPXDVZuZQYKj9TnmxUq7Qcu1K2Avlr+obIFSLKuphKqAxMz/PqqJTCRmdmd9yyLZTBorLzitoq03S1YBDJ/4LEUwHyxr5avbtxirbZ6mDfrMwFOl99Zsjnbm3jaaEvjVVVXNEQo5AKn68quZbFV33nZV9BukAOpWxiZeS8ZayU41/lZp0HWntTl+ltyo1q2tc/NyjBec7Icir8SOZHY2gZUXnFaXnekkkqSJcMtywy2y8gd7mxrEAbVP3Ezw2ZH+EH52yU+xfdkr+GTnXdrkoMPcpW7PFsTTrFvpFo4DSNXaAc2LxFM3xxTEgbqVscmSGxG4QYUqUNad1iYlebH8RWULdLMfCrEijSUagKqq4S3Lz6h5zCk4/03/mVi2oBcLbn4ChWJ8ekpnJuW5QQUKqDhc3sY0bCj0458mnYMOBp558O9wQ/uD2ImDoMlTgXx7jasjhw7cNPY53J75lvoEfr/8TivdwnEAJaNYlTU4xRlgdJLO/H7gree8u1aA6qaml/mqzlup7eKA7hg/m5e6/4Ok+eXHKbIijxldPXEANdmavZqkoKmdmYrlUFePJSq8BHETs77K6aXv44VlP8U5n/gKThRKOHv0GdyS2YyTccCoOZ47bOgollXxy+/7K+xIf0ifyu/3y+8mA6hW+XEGmPwxvTZ80a3GRcWu+2tX6mSM1eiA6FfzlszNpiKBPGZ0DhV7fRWV3JLNpLHm4tM9nacz0/j/SrPuy7IFvZXXqaqqiFLBcFqsHQaufRnvX/qnuGX5Gdjcfnl9Kn+QL//8fvcCUvYV7ZIbjSQi39iyN1XVE4t5o3a46Z6xO2qAqmvk2peN4N53heZ8bFyEvJQC8Jv274TXBhHma9u6wkhcUslYQuyItOITlUxizbi0o3Oo2B83x3AauzubwXBOvSqfOrkDxWOjGB3ztqq+/ZPvxbUPvFh3g29vNqFKCpramcGeG89Xvh7t5ubRgZo157IFvVi24CZg3+nReI8vXO+8GWhf0ZrnsGdYOvnJKQ1ceqeikYKGo/uNWuXWBBxdH82LbjWcM8pxBrzJMlGu2r1o3vYN2Nxh42Jjl7GE2JFA7gN7Io9VJtEFc53+raqvYgQ3/UVh7dLTsfLBvSgofIqDwzlkUoRMmjzp6Bsff7VubvYU+1lktHpDAZVgrrtLMMfSOVr+H6Yr04M9abhe0AVmQL/Kd9KGVRcFLtYHYTcniqpWuc7NMWW2s8782hPQBnGvdzJedW8vmrc4VRKDSCs+8CqTWNFJJisvOM33+Zct6MXGT5yp1dMLJcaYmxm9zNBwDue8Z0bNYypZpJPyWNVm6KIE4GMLay82ZmPoweEcCHpHyy35T3iaVyjm9xsFqpbfFc6FYTo5VD0y7fJC0GYPqhWvm87stOF55mXur9GP7u1F89au2veLf7zByIrcB15lEiteJBM/mKt2Xa0VrxUXZnZn8cwvD9Q+ppFFZtIhY2yg5jn2OxRGvaPFrNmy+10f9jaxKIhilT+/v5xOr8AawMzzPHWzP4+4SgapGUuxYnZa/ZtNmp3ws4J2m4vbfFTykRAbEsh94EUmcdLQzcfNFbxTMFeNY44R1kdu3hFc+8CLNY/rZBFrKVvrRUt1hwIAPyotxvZ8VVfPZtK4RXUH4sfeFncdD9X4TvKC6njAY/9OTcIQ4HwRWnKjvgGzF3uiX6+32wVRlUxkIhJLQxFpxQduMonOaviX217SPr5o3dOYu/pRLFr3NLbtGdSOs/IHe7HywWDJQFM7M+jOZmo6DC1b0Fun0zsl+phYn6O7E+Hyeeznq8HPbX7Y9G+31ma68U89Xy0vnHq++vi3nvN4S6RJGHLDyZ3jZaMz6i49pgSlQ5KBGoasyH3gJpPoNHRVedpcoYj7nnurIo9YN05V4wRNBFI5TExWXnBajTSyvbQY7ZzCzZ0PIZv7DYZ4GtYX+ms2Oq3avu4OxVqlUYuf2/wwm2r2jUuVa0Q3/mtPGIHKvvLWHb/7O94KdVl7d+rmrLv7ULlzvG50RpWOb8XMkpVkoKYigdwnTs4S3QpVV2Pc/qi5ceqkuTuhcqwcOzGGbXsG6+ZsSje5QhFpIhSZ0dudxeILrkLngr8GALywZxC7H38VpNH27RcCwMdGrp/b/DDp37qg+/CXDA3cSec1bX/2i4VOO/dUbZGM8902Ty0PuV14vGjXOkwLo3nBobS3TVI34rhACL6QQO6AX8+4boVqBkovBNW/u7NGcovdZ14oMb62ZS+A6h2FfZOyyFwJwNbX52aHNH/34qObcGX+e5iZOoQT2ZPRmb4ZgEtw8JPSHSb9WxfszaB7dD8MP46qA5BGC9fNh9LqYF553HIenZ/cy92HNaCb8zMvSm77DHvvr86Ri8bPc84KF8zDXFyESBCNXINO7zZ1bBU6Df3TH5xd93jUEAFHNclCReaauQexUepYln4Wa+nbmJU6iBTYqGDoRb/2k9IdJv3b0+09oy5F3kkL12nnC/9E/fild5a1bdvFQpUp6efuw+/egddszSBY66Vf+7IE8QYjgVxDkGC3bEEvbll+Rt1G3zeXnVHzeBwMjxS0TZyB2rkHsVFqCRoc/FTdC1Ohz7PPm+vHf+0J9Wt75WEjHd0k22Mcf9Gt6nm+9Zy+o489QPvZkPT73kuFwnGLSCsaggY7nRxhfTyONm2m9OPUxLmSRu8j29SVMMHBj987qDfcftvvp7WZTgu3B2VrZyFznhXJQ2MXrJzXFqD96M1+33upUDhukRW5Bl1QCxTsbKgkmLAcHx0DANyy/AykNS15zLlHmW0auaUtDqy3/Zfe6V2m8foa7KvgR64zLgJeEoTs5/V697Fvi7pgl9O8pULhuEUCuYYog52Zxm76xQGEllrs/T2Hc4WKffFv+s90nLtOAgqUbdpqwcEpUNr95iotXIe5Ct63xV+9cd0cnfRmUxtX3Vk4vfdRNZEQEgex15zuCOnr6+Ndu3Y1/Lx+8eta0Y1hlztM70JveUy/2ZqZNGFye5uyEqLp4Y5i7p4ZD93TVYWyMlnDnvfaE9Xu9SeG1Q2LTWnGsaemArdu9ar3VufbVlVnFMYVRLSbmfvqHo8ikBPR1wFsBDCDmdUFOyy0SiCPAjc9PJtJ42MLe/HQ7kGttm0nkyJlBUTAuEi8se6jQaY6sdEFYDPQOvXMzGSrK9u13fC3Gidj5a1Cd3HRtnxzGEsYF+gCeWhphYhmA/gwgLfCjpUU7FKIk+XQDbeVdq5QxDO/POCobdsplFh7bIoo1HwThVtqfZS4bRzqemZSulae8Ls34HS8zpWiqsro5dyNfD+FhhKFa+U2AKsA/DCCsZpOkJrj1uda5Yxz3jNDl2pSw9BwrjL2yh/s9ZSOX2RWjm16xr3M1ysNlWlMvKTWR4nO0WF2sdfJJVysnY+ykBQBc88GBn7mL/vRKZnJvjK3j2WXZE4930j+adT72QjGg6QXEaFW5ES0FMAgM+/1cOwKItpFRLsOHDjgdnjTCJoso0ogstZSccJ0kyxb0IuNHz8TUzurLcic1ui6sYMm96gIkhgVCXEmr6jQtX4z+2/qVsH2x1Ubiss3AZ/f7n+jUesImu08lipRaNc96vfzsetbc5UetpDaOMNVIyeiJwFlc5dvAPgLAOcz81Ei+jWAvlbXyHV1vt2056De8Gwm7egY2bZnUNsVyImotHLd6/JUGCsMWq05Rh14/Vx14o6uc4/J2qPxzEenkTtdAPZtMerIeKr7osBt/KTgtqcxTgmskTPzecw8z/4HwOsA5gLYWw7iswD8nIiUHb1ahaD+cT9ZkWkif7a/AB7FKPzuQMRZoH5ohj9d17z46IC+YqFbJcMw+LULOtkSvRLnXU+USJZqDYE1cmZ+CcC/MX/2syJPMkEr+umyJe06ttsK3M7Gx18NVMI2UHKPgkizQP3QjIp6TpmPzarw5yejVbchW8HLjg1aIxhKlmoNkhBUxnSqXPvAi+hoS2FqZ30jBid0CUSfOWtOqMSboI0k3LoPeXXlRJoF6gc/q9Go3BhOyU2tkEzjFIAzWaDviur8sz3+M0MbjdP/a6slosVMZLVWmPmUqMZqNHanynCugGwmjds++V7PQTfq3pzmvDyuoSroutxbx/TjyonjdXnGy2o0SneLWznWKHqBxolTeV3VZqgqscktGDbKKRJnXfZxiGR2ookbei543UC1Z4o6BdmkvtbATNBNLyVeN0d175lbZmiQzdegyP+rEt1mp1Q/RBM39Fzwen4ziHsJxEl9rYGRTa8qXlepWn96yTkgh2m55xf5f/WFBHKE39CLK2FGNy8VXgNx0zYv46JVNr0aJUl4kX+CvmeNDK6t8v+aECb8Zue2PYMYyY/VPe51Qy/OhBnVRqPOieg1EDdt8zIu/Gx6NStFPWnJK0E3ChtpCZXNTF9M6EBuBuEjI7VVBLuzGc/ukijbptlRlZv9zFlzQgXiSEvYJgE/9bubFUwbnaXqRlAHTiODayu4hBLEhN7sjGLjL2gmaBiaUvskQecPhNfNszgkkGZkqcaF1DcJTgTvnWx2KtDpz368283QnN2628dJmKJiTcWLvhtXoa7xpPcm3YKZVGIuAjehpRVdKViv5WSBcag5uxCnlBQrXvTduCQQ0XuFmOW1CR3IixpZSfe4inGnObvQsvZFL8E0LleG6L1CzI6fCS2t9GpkkV6fskgzpY5G07L2RS8e6zglEJEkJjYxy2sTekU+0WSRKGjp98ytqbFIIEJcxPzZmtAr8qbWEWlRxvV7JvU7hLiI+bM1oe2HgiAIrURszZcFQRCE5iKBXBAEocWRQC4IgtDiSCAXBEFocSSQC4IgtDihAzkR/TkRvUpErxDRhigmJQiCIHgnlI+ciM4BcAmA+cw8SkT/JpppCYIgCF4JuyL/MwDrmHkUAJj5t+GnJAiCIPghbCD/AwB/RETPE9FPiOj9ugOJaAUR7SKiXQcOHAh5WkEQBMHEVVohoicBnKz41TfKz58K4CwA7wewhYjezYp0UWbeBGATYGR2hpm0IAiCUMU1kDPzebrfEdGfAdhaDtw/I6ISgOkAZMktCILQIMIWzdoG4FwAPyaiPwDQDuBg2EnFSUu2KRMEN6QF24QmbCC/B8A9RPQygDyAz6tklaTQsm3KBMGJmNuICckn1GYnM+eZ+XJmnsfM72Pmp6OaWBy0bJsyQXAi5jZiQvKZUJmdLdumTBCciLmNmJB8JlQg17UjS3ybMkFwwktjaWFcM6ECeUu3KRMEHdKibsIzoVq9jes2ZcLERVrUTXik1ZsgCEKLIK3eBEEQxikSyAVBEFocCeSCIAgtjgRyQRCEFkcCuSAIQovTFNcKER0A8GbDT1xlOhJe3Asyx6iQOUaDzDEaws7x95l5hv3BpgTyZkNEu1QWniQhc4wGmWM0yByjIa45irQiCILQ4kggFwRBaHEmaiDf1OwJeEDmGA0yx2iQOUZDLHOckBq5IAjCeGKirsgFQRDGDRLIBUEQWpwJGciJ6K+IaB8RvUhETxDRzGbPyQ4RbSSiX5bn+TARdTd7TnaI6BNE9AoRlYgoUbYvIvoIEb1KRP9KRKubPR8VRHQPEf223PM2cRDRbCJ6hoh+Uf5/vqbZc7JDRJOI6GdEtLc8x5uaPScdRJQmoj1E9EjUY0/IQA5gIzPPZ+b3AngEQBIr8O8AMI+Z5wP4FYAbmjwfFS8DWA7gp82eiBUiSgP4ewAXAvhDAJ8moj9s7qyUfAfAR5o9CQfGAHyNmf8DgLMAfDmB7+MogHOZ+UwA7wXwESI6q7lT0nINgF/EMfCEDOTM/DvLj5MBJG7Hl5mfYOax8o/PAUhc3y5m/gUzJ7Fz9QcA/Cszv87MeQDfB3BJk+dUBzP/FMDhZs9DBzO/zcw/L//7HRhBKFFdWNjgWPnHTPlP4r7PRDQLwEcBbI5j/AkZyAGAiP6aiPYD+AySuSK3cgWAx5o9iRaiF8B+y88DSFgAajWI6BQACwA83+Sp1FGWLF4E8FsAO5g5cXMEcDuAVQBKcQw+bgM5ET1JRC8r/lwCAMz8DWaeDeA+AF9J4hzLx3wDxi3ufUmdYwIhxWOJW6W1CkTUBeAhAF+13c0mAmYulmXSWQA+QETzmjylGojoIgC/ZebdcZ1j3PbsZObzPB56P4BHAayJcTpK3OZIRJ8HcBGAJdwkw7+P9zFJDACYbfl5FoChJs2lpSGiDIwgfh8zb232fJxg5mEi+jGMfYckbSAvArCUiP4YwCQA7yKi7zHz5VGdYNyuyJ0golMtPy4F8MtmzUUHEX0EwPUAljLzSLPn02K8AOBUIppLRO0APgVge5Pn1HIQEQG4G8AvmPnWZs9HBRHNMB1dRJQFcB4S9n1m5huYeRYznwLjs/h0lEEcmKCBHMC6sjywD8D5MHaTk8bfATgJwI6yTfLOZk/IDhFdSkQDAP4jgEeJ6PFmzwkAypvEXwHwOIwNui3M/EpzZ1UPEf0DgH8GcBoRDRDRF5o9JxuLAHwWwLnlz+CL5VVlkvg9AM+Uv8svwNDII7f3JR1J0RcEQWhxJuqKXBAEYdwggVwQBKHFkUAuCILQ4kggFwRBaHEkkAuCILQ4EsgFQRBaHAnkgiAILc7/B9TJ4teKaHCtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 查看源数据分布\n",
    "plt.scatter(df[\"X1\"],df[\"y\"])\n",
    "plt.scatter(df[\"X2\"],df[\"y\"]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:39:19.440462Z",
     "start_time": "2024-03-04T07:39:19.432056Z"
    }
   },
   "outputs": [],
   "source": [
    "# 切分特征和标签\n",
    "X = df.iloc[:,:-1]\n",
    "y = df.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:39:38.900192Z",
     "start_time": "2024-03-04T07:39:38.878284Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.33</td>\n",
       "      <td>-0.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.32</td>\n",
       "      <td>-0.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.07</td>\n",
       "      <td>-1.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.56</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.63</td>\n",
       "      <td>0.22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>-0.26</td>\n",
       "      <td>1.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>0.60</td>\n",
       "      <td>-1.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>1.55</td>\n",
       "      <td>-0.36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>-1.36</td>\n",
       "      <td>-0.61</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>-1.44</td>\n",
       "      <td>0.50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       X1    X2\n",
       "0    1.33 -0.77\n",
       "1   -0.32 -0.99\n",
       "2   -1.07 -1.44\n",
       "3    0.56  0.30\n",
       "4   -1.63  0.22\n",
       "..    ...   ...\n",
       "995 -0.26  1.14\n",
       "996  0.60 -1.11\n",
       "997  1.55 -0.36\n",
       "998 -1.36 -0.61\n",
       "999 -1.44  0.50\n",
       "\n",
       "[1000 rows x 2 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:39:34.376532Z",
     "start_time": "2024-03-04T07:39:34.364554Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      4.44\n",
       "1      1.38\n",
       "2      0.30\n",
       "3      1.82\n",
       "4     -2.48\n",
       "       ... \n",
       "995   -0.66\n",
       "996    3.28\n",
       "997    4.46\n",
       "998   -1.11\n",
       "999   -2.38\n",
       "Name: y, Length: 1000, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:39:50.633284Z",
     "start_time": "2024-03-04T07:39:50.623555Z"
    }
   },
   "outputs": [],
   "source": [
    "# 切分训练集和测试集\n",
    "Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:40:29.061938Z",
     "start_time": "2024-03-04T07:40:29.039382Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 建立线性回归模型\n",
    "reg = LinearRegression().fit(Xtrain, Ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:40:30.718819Z",
     "start_time": "2024-03-04T07:40:30.708112Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression()"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:40:41.106170Z",
     "start_time": "2024-03-04T07:40:41.085130Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.53896706,  1.50978455, -3.64873645,  0.45022645,  1.78970907,\n",
       "       -0.53971692, -0.25964424, -1.41931553, -2.28900294,  1.72926474,\n",
       "        3.83881812,  3.14910222,  0.57002191, -0.85939841, -0.12965725,\n",
       "       -2.22903513,  2.74920603,  2.22933419,  0.66019587,  2.65937645,\n",
       "        4.59854918, -6.41780759, -0.49968766,  2.82952885,  2.86922975,\n",
       "        2.16980286, -1.02938262, -1.00968033,  4.33887155,  1.34960343,\n",
       "       -1.67940162, -1.68936088,  1.56972433,  1.1601532 ,  3.91888865,\n",
       "       -0.79949868,  0.42030061, -0.48977245,  1.1795992 ,  3.8588848 ,\n",
       "        1.31956146,  3.30905108,  3.14936651, -4.49806885,  0.29023354,\n",
       "       -0.09974343, -0.73946279,  6.54783735, -1.84912955,  0.51997332,\n",
       "        2.50932679,  1.53007949, -2.12942648,  6.18833483,  3.28956102,\n",
       "        6.03834523,  1.33998454, -1.38915343, -0.24973703,  3.9590861 ,\n",
       "        5.34860931,  2.08929184, -3.05856112,  1.59946196, -2.61885191,\n",
       "        2.98961386,  1.27948014,  3.24923143,  4.61898828,  0.13011649,\n",
       "        0.7698719 ,  1.39976415,  3.0793193 ,  7.25792827, -4.53825829,\n",
       "        3.92940453,  4.22896727,  0.55996655,  0.15028729,  3.25920671,\n",
       "       -1.8589847 ,  3.71914679,  3.24923944, -0.67952702, -2.16888311,\n",
       "        1.60958941, -1.81959214,  4.59889356, -3.4890719 ,  0.02994323,\n",
       "        1.34961945,  6.54821777,  2.17931763,  2.75945361,  3.84910974,\n",
       "        0.04010271, -3.59868786, -1.30950336,  0.6703073 ,  0.67007104,\n",
       "       -0.12003836,  2.58889677,  2.92921758,  0.96957372,  3.72904598,\n",
       "        2.39969882,  0.82005263,  1.02997801, -0.01970492, -1.0796034 ,\n",
       "        2.18961327, -4.15840076,  0.68982939, -4.10862047,  0.75005749,\n",
       "       -1.62947316, -0.98968172, -2.81881002, -0.63931355, -2.38897598,\n",
       "       -0.70948089, -1.25929059,  0.02997126,  2.21970729, -0.44968712,\n",
       "       -1.06948396, -1.6192376 , -0.45952625, -1.57930845,  2.57941804,\n",
       "       -1.10928497, -2.26911244, -1.19971521,  1.2897998 ,  0.02017618,\n",
       "        1.83956545, -0.45949021,  2.83961225, -0.64932087,  0.41008907,\n",
       "        1.18975068,  0.35020936, -3.90867438,  4.28874287, -0.14960381,\n",
       "        3.09943805,  3.56906509,  0.87982423,  0.9701023 , -2.24904175,\n",
       "        1.41986287, -0.17983398,  0.62011455, -0.70943684,  1.01988259,\n",
       "        0.98971248,  1.44007372,  0.21998211,  0.86024207,  0.90994228,\n",
       "        0.42981939,  2.24948897, -0.509715  ,  0.54010408,  2.98909729,\n",
       "        4.05877083,  2.16963467,  0.48002014,  1.03031438, -0.78951138,\n",
       "        0.53994791,  0.23016962,  1.75973118,  1.1698802 , -1.38956989,\n",
       "        5.29855271,  5.76835594,  4.00878631,  0.86977687,  0.67970995,\n",
       "        0.25000406, -2.3790928 ,  0.32022745, -1.46950428,  0.57027019,\n",
       "        2.65928435,  2.88928843,  0.01022492,  6.33852465, -3.25880754,\n",
       "        1.91978015, -1.38929759,  0.53005272,  0.47041727,  1.3795533 ,\n",
       "       -2.00925861,  1.51960366, -1.86912416,  1.61974088,  2.59923646,\n",
       "        0.65007643,  2.18941305,  2.31958423,  8.0674076 ,  1.54973372,\n",
       "       -1.14958653,  3.54896236, -1.6792935 ,  1.11004454, -0.49967164,\n",
       "       -2.71887702,  0.29983641, -0.85929029, -3.51860131, -0.11998631,\n",
       "       -0.24960088,  1.84938456,  2.01949691, -2.019338  ,  3.72900194,\n",
       "        2.2096279 ,  1.25005085, -1.6796499 , -0.93953702,  1.29993526,\n",
       "        1.18993888,  2.11981834,  0.49007952,  0.11026204,  0.53967161,\n",
       "        1.96957646,  1.58999925, -1.56938522,  1.0899338 , -0.94954834,\n",
       "       -1.40937229,  2.25936014,  4.21900801,  2.17958192,  4.36878137,\n",
       "       -1.08915822,  0.25998334, -0.76931655,  5.80825706,  2.88904816,\n",
       "       -0.87951716, -0.46963768,  0.29979636, -1.69922805,  2.7892393 ,\n",
       "       -0.78928313,  2.32983982, -1.17914797,  1.8797629 ,  4.98877442,\n",
       "        2.82946077,  1.98967919,  0.58016137,  3.79909719,  2.32954349,\n",
       "        2.39921028,  2.07974503, -0.78943129,  4.4989085 ,  5.77802688,\n",
       "       -0.18986132, -2.98868611,  0.89011185, -2.36900139,  7.27813912,\n",
       "       -0.52956945, -4.87849099, -0.09963931,  1.97938756, -1.4889663 ,\n",
       "       -0.30961674, -0.62973871,  2.14942383, -5.0176764 , -0.36950046,\n",
       "        3.79922133, -0.45984661,  4.27884767,  1.36007526, -1.52956819,\n",
       "       -2.67871561,  1.71966588, -1.79936527,  0.62998972,  3.30895097,\n",
       "       -0.12998561, -4.51864009,  2.16945848,  0.65997563,  3.2191334 ,\n",
       "        3.51920871, -1.10965738,  0.13009647,  1.34970354, -2.23911853])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat = reg.predict(Xtest)\n",
    "yhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:40:55.445655Z",
     "start_time": "2024-03-04T07:40:55.438233Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(300,)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:41:07.563363Z",
     "start_time": "2024-03-04T07:41:07.554930Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(300, 2)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtest.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 2)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T07:41:23.064565Z",
     "start_time": "2024-03-04T07:41:23.048799Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.99946089, -0.99953023])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 探索建立好的模型\n",
    "reg.coef_ #查看回归系数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9998959940347021"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.intercept_ #查看截距"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 属性           | 含义                                                         |\n",
    "| -------------- | ------------------------------------------------------------ |\n",
    "| **coef_**      | 数组，形状为 (n_features, )或者(n_targets, n_features)<br>线性回归方程中估计出的回归系数。如果在fit中传递多个标签（当y为二维或以上的时候），则返回的系数是形状为（n_targets，n_features）的二维数组，而如果仅传递一个标签，则返回的系数是长度为n_features的一维数组 |\n",
    "| **intercept_** | 数组，线性回归中的截距项。                                   |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "至此，我们就通过建模得到了这样的一个回归方程：$ y = 2X_1-X_2+1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.95905334, 0.        ])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 尝试改变positive参数的值（强行将回归系数定为正数）\n",
    "reg1 = LinearRegression(positive=True).fit(Xtrain, Ytrain)\n",
    "reg1.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9759123645888331"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg1.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果上面代码运行不了，尝试检查一下sklearn的版本，只有0.24及以上版本才有此参数。若sklearn版本低于0.24，用以下方法将sklearn升级到0.24版本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.24.2'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sklearn\n",
    "sklearn.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "先升级pip:`python -m pip install --upgrade pip`<br>\n",
    "再使用pip升级sklearn到指定版本：`pip install --user --upgrade scikit-learn==0.24`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5  回归类的模型评估指标"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "回归类算法的模型评估一直都是回归算法中的一个难点，但不像我们曾经讲过的无监督学习算法中的轮廓系数等等评估指标，回归类与分类型算法的模型评估其实是相似的法则——找真实标签和预测值的差异。只不过在分类型算法中，这个差异只有一种角度来评判，那就是是否预测到了正确的分类，而在我们的回归类算法中，我们有两种不同的角度来看待回归的效果：\n",
    "\n",
    "第一，我们是否预测到了正确的数值。\n",
    "\n",
    "第二，我们是否拟合到了足够的信息。\n",
    "\n",
    "这两种角度，分别对应着不同的模型评估指标。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 是否预测了正确的数值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "回忆一下我们的残差平方和SSE，它的本质是我们的预测值与真实值之间的差异，也就是从第一种角度来评估我们回归的效力，所以SSE既是我们的损失函数，也是我们回归类模型的模型评估指标之一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但是，SSE有着致命的缺点：它是一个无界的和，可以无限地大。我们只知道，我们想要求解最小的SSE，从SSE的公式来看："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$SSE=\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "它不能为负，所以SSE越接近0越好，但我们没有一个概念，究竟多小才算好，多接近0才算好？为了应对这种状况，sklearn中使用SSE的变体，均方误差MSE（mean squared error）来衡量我们的预测值和真实值的差异："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$MSE = \\frac{1}{n}\\sum_{i=1}^{n}({y_i} - {\\hat{y}_i})^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "均方误差，本质是在SSE的基础上除以了样本总量，得到了每个样本量上的平均误差。在sklearn当中，我们有两种方式调用这个评估指标，一种是使用sklearn专用的模型评估模块metrics里的类mean_squared_error，另一种是调用交叉验证的类cross_val_score并使用里面的scoring参数来设置使用均方误差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:07:08.818614Z",
     "start_time": "2024-03-04T08:07:08.801120Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error as MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:07:10.191827Z",
     "start_time": "2024-03-04T08:07:10.181482Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.00015119928019778137"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MSE(Ytest,yhat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:15:59.133403Z",
     "start_time": "2024-03-04T08:15:59.064733Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.00016865, -0.0001646 , -0.00014116, -0.00013906, -0.00010599,\n",
       "       -0.00017468, -0.00014607, -0.00011486, -0.00015278, -0.00014928])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_score(reg,X,y,cv=10,scoring=\"neg_mean_squared_error\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\" size=3>**注意：在交叉验证中，均方误差为负。**</font><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " 这个该如何解释呢？从公式来看，均方误差的值永远为非负，但是sklearn中的交叉验证参数scoring下，均方误差作为评判标准时，却是计算”负均方误差“（neg_mean_squared_error）。这是因为sklearn在计算模型评估指标的时候，会考虑指标本身的性质，均方误差本身是一种误差，所以被sklearn划分为模型的一种损失(loss)。在sklearn当中，所有的损失都使用负数表示，因此均方误差也被显示为负数了。真正的均方误差MSE的数值，其实就是neg_mean_squared_error去掉负号的数字。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了MSE，我们还有与MSE类似的MAE（Mean absolute error，绝对均值误差）："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$MAE = \\frac{1}{n}\\sum_{i=1}^{n}|{y_i} - {\\hat{y}_i}|$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.008973153808188776"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import mean_absolute_error as MAE\n",
    "MAE(Ytest,yhat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:16:15.722999Z",
     "start_time": "2024-03-04T08:16:15.653505Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.00887854, -0.00950097, -0.00888558, -0.00869893, -0.00738316,\n",
       "       -0.01040722, -0.00869176, -0.00776789, -0.00987094, -0.0086515 ])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_score(reg,X,y,cv=10,scoring=\"neg_mean_absolute_error\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['accuracy',\n",
       " 'adjusted_mutual_info_score',\n",
       " 'adjusted_rand_score',\n",
       " 'average_precision',\n",
       " 'balanced_accuracy',\n",
       " 'completeness_score',\n",
       " 'explained_variance',\n",
       " 'f1',\n",
       " 'f1_macro',\n",
       " 'f1_micro',\n",
       " 'f1_samples',\n",
       " 'f1_weighted',\n",
       " 'fowlkes_mallows_score',\n",
       " 'homogeneity_score',\n",
       " 'jaccard',\n",
       " 'jaccard_macro',\n",
       " 'jaccard_micro',\n",
       " 'jaccard_samples',\n",
       " 'jaccard_weighted',\n",
       " 'max_error',\n",
       " 'mutual_info_score',\n",
       " 'neg_brier_score',\n",
       " 'neg_log_loss',\n",
       " 'neg_mean_absolute_error',\n",
       " 'neg_mean_absolute_percentage_error',\n",
       " 'neg_mean_gamma_deviance',\n",
       " 'neg_mean_poisson_deviance',\n",
       " 'neg_mean_squared_error',\n",
       " 'neg_mean_squared_log_error',\n",
       " 'neg_median_absolute_error',\n",
       " 'neg_root_mean_squared_error',\n",
       " 'normalized_mutual_info_score',\n",
       " 'precision',\n",
       " 'precision_macro',\n",
       " 'precision_micro',\n",
       " 'precision_samples',\n",
       " 'precision_weighted',\n",
       " 'r2',\n",
       " 'rand_score',\n",
       " 'recall',\n",
       " 'recall_macro',\n",
       " 'recall_micro',\n",
       " 'recall_samples',\n",
       " 'recall_weighted',\n",
       " 'roc_auc',\n",
       " 'roc_auc_ovo',\n",
       " 'roc_auc_ovo_weighted',\n",
       " 'roc_auc_ovr',\n",
       " 'roc_auc_ovr_weighted',\n",
       " 'top_k_accuracy',\n",
       " 'v_measure_score']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sklearn\n",
    "sorted(sklearn.metrics.SCORERS.keys()) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 是否拟合了足够的信息"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于回归类算法而言，只探索数据预测是否准确是不足够的。除了数据本身的数值大小之外，我们还希望我们的模型能够捕捉到数据的”规律“，比如数据的分布规律，单调性等等，而是否捕获了这些信息并无法使用MSE来衡量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"https://pictes.oss-cn-beijing.aliyuncs.com/%E5%BE%AE%E8%AF%BE%20-%20sklearn/week%209%20Linear%20Model/MSE.PNG\" style=\"zoom:60%;\"><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "来看这张图，其中红色线是我们的真实标签，而蓝色线是我们的拟合模型。这是一种比较极端，但的确可能发生的情况。这张图像上，前半部分的拟合非常成功，看上去我们的真实标签和我们的预测结果几乎重合，但后半部分的拟合却非常糟糕，模型向着与真实标签完全相反的方向去了。对于这样的一个拟合模型，如果我们使用MSE来对它进行判断，它的MSE会很小，因为大部分样本其实都被完美拟合了，少数样本的真实值和预测值的巨大差异在被均分到每个样本上之后，MSE就会很小。但这样的拟合结果必然不是一个好结果，因为一旦我的新样本是处于拟合曲线的后半段的，我的预测结果必然会有巨大的偏差，而这不是我们希望看到的。所以，我们希望找到新的指标，除了判断预测的数值是否正确之外，还能够判断我们的模型是否拟合了足够多的，数值之外的信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们讲了一个公式：$SST=SSE+SSR$，根据这个公式我们衍生出了一个新的指标：**$R^2$**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ R^2 = \\frac{SSR}{SST} = \\frac{SST-SSE}{SST} = 1- \\frac{SSE}{SST}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/fig-ch12_04_05m.jpg\" style=\"zoom:60%;\"><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 总离差平方和SST反映的是真实标签所带的信息量，在$R^2$中，分子是真实值和预测值之差的差值，也就是我们的模型没有捕获到的信息总量，分母是真实标签所带的信息量，所以其衡量的是**1 - 我们的模型没有捕获到的信息量占真实标签中所带的信息量的比例**，所以，$R^2$越接近1越好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$R^2$可以使用三种方式来调用，一种是直接从metrics中导入r2_score，输入预测值和真实值后打分。第二种是直接从线性回归LinearRegression的接口score来进行调用。第三种是在交叉验证中，输入\"r2\"来调用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999736813282624"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import r2_score\n",
    "r2_score(Ytest,yhat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999736813282624"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2 = reg.score(Xtest,Ytest)\n",
    "r2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:27:31.577182Z",
     "start_time": "2024-03-04T08:27:31.508522Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.99996887, 0.99996604, 0.99997416, 0.99997008, 0.99997867,\n",
       "       0.99996775, 0.99996424, 0.99997402, 0.99996714, 0.99996825])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_score(reg,X,y,cv=10,scoring=\"r2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6 【案例】使用线性回归预测鲍鱼的年龄"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> UCI数据集官网：http://archive.ics.uci.edu/ml/datasets.php"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此案例所用数据集来自UCI数据集，记录了鲍鱼（一种介壳类水生生物）的一些相关属性，根据这些属性来预测鲍鱼的年龄。鲍鱼年龄的计算主要是通过一些比较容易获得的量测数据，以及通过锥体切割贝壳，染色并在显微镜下数出环数来确定的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"https://typora-photo1220.oss-cn-beijing.aliyuncs.com/DataAnalysis/image-20220114112711771.png\" style=\"zoom:60%;\"><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|特征名| 数据类型| 单位| 描述|\n",
    "| -----|-------|----- | ------- |\n",
    "|性别 |离散型| —— |公，母，婴儿（1,-1,0）|\n",
    "|长度 |连续型| 毫米| 贝壳最长的部分|\n",
    "|直径 |连续型 |毫米| 垂直于长度|\n",
    "|高度| 连续型 |毫米| 壳里的肉的高度|\n",
    "|整体重量| 连续型 |克 |整个鲍鱼的重量|\n",
    "|肉重量 |连续型| 克 |鲍鱼肉的重量|\n",
    "|内脏重量 |连续型 |克 |内脏的重量 (去血后)|\n",
    "|壳重 |连续型 |克 |干了之后的壳重|\n",
    "|年龄| 整数型| ——| 鲍鱼的年龄|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:30:54.602837Z",
     "start_time": "2024-03-04T08:30:54.589780Z"
    }
   },
   "outputs": [],
   "source": [
    "# 导入所需模块和包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:31:16.652593Z",
     "start_time": "2024-03-04T08:31:16.624867Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0      1      2      3       4       5       6      7   8\n",
       "0  1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1  1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2 -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3  1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4  0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入数据集\n",
    "data = pd.read_table('abalone.txt',header=None)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:31:35.449274Z",
     "start_time": "2024-03-04T08:31:35.434261Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4177, 9)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:32:20.801994Z",
     "start_time": "2024-03-04T08:32:20.794217Z"
    }
   },
   "outputs": [],
   "source": [
    "# 简单数据处理及探索\n",
    "data.columns=['性别','长度','直径','高度','整体重量','肉重量','内脏重量','壳重','年龄']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:32:25.637098Z",
     "start_time": "2024-03-04T08:32:25.613445Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "      <th>年龄</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.1500</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.0700</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.2100</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.1550</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.0550</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4172</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.450</td>\n",
       "      <td>0.165</td>\n",
       "      <td>0.8870</td>\n",
       "      <td>0.3700</td>\n",
       "      <td>0.2390</td>\n",
       "      <td>0.2490</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4173</th>\n",
       "      <td>1</td>\n",
       "      <td>0.590</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.9660</td>\n",
       "      <td>0.4390</td>\n",
       "      <td>0.2145</td>\n",
       "      <td>0.2605</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4174</th>\n",
       "      <td>1</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.475</td>\n",
       "      <td>0.205</td>\n",
       "      <td>1.1760</td>\n",
       "      <td>0.5255</td>\n",
       "      <td>0.2875</td>\n",
       "      <td>0.3080</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4175</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.625</td>\n",
       "      <td>0.485</td>\n",
       "      <td>0.150</td>\n",
       "      <td>1.0945</td>\n",
       "      <td>0.5310</td>\n",
       "      <td>0.2610</td>\n",
       "      <td>0.2960</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4176</th>\n",
       "      <td>1</td>\n",
       "      <td>0.710</td>\n",
       "      <td>0.555</td>\n",
       "      <td>0.195</td>\n",
       "      <td>1.9485</td>\n",
       "      <td>0.9455</td>\n",
       "      <td>0.3765</td>\n",
       "      <td>0.4950</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4177 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      性别     长度     直径     高度    整体重量     肉重量    内脏重量      壳重  年龄\n",
       "0      1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.1500  15\n",
       "1      1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.0700   7\n",
       "2     -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.2100   9\n",
       "3      1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.1550  10\n",
       "4      0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.0550   7\n",
       "...   ..    ...    ...    ...     ...     ...     ...     ...  ..\n",
       "4172  -1  0.565  0.450  0.165  0.8870  0.3700  0.2390  0.2490  11\n",
       "4173   1  0.590  0.440  0.135  0.9660  0.4390  0.2145  0.2605  10\n",
       "4174   1  0.600  0.475  0.205  1.1760  0.5255  0.2875  0.3080   9\n",
       "4175  -1  0.625  0.485  0.150  1.0945  0.5310  0.2610  0.2960  10\n",
       "4176   1  0.710  0.555  0.195  1.9485  0.9455  0.3765  0.4950  12\n",
       "\n",
       "[4177 rows x 9 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4177, 9)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:32:34.329948Z",
     "start_time": "2024-03-04T08:32:34.298715Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4177 entries, 0 to 4176\n",
      "Data columns (total 9 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   性别      4177 non-null   int64  \n",
      " 1   长度      4177 non-null   float64\n",
      " 2   直径      4177 non-null   float64\n",
      " 3   高度      4177 non-null   float64\n",
      " 4   整体重量    4177 non-null   float64\n",
      " 5   肉重量     4177 non-null   float64\n",
      " 6   内脏重量    4177 non-null   float64\n",
      " 7   壳重      4177 non-null   float64\n",
      " 8   年龄      4177 non-null   int64  \n",
      "dtypes: float64(7), int64(2)\n",
      "memory usage: 293.8 KB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "      <th>年龄</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "      <td>4177.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.052909</td>\n",
       "      <td>0.523992</td>\n",
       "      <td>0.407881</td>\n",
       "      <td>0.139516</td>\n",
       "      <td>0.828742</td>\n",
       "      <td>0.359367</td>\n",
       "      <td>0.180594</td>\n",
       "      <td>0.238831</td>\n",
       "      <td>9.933684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.822240</td>\n",
       "      <td>0.120093</td>\n",
       "      <td>0.099240</td>\n",
       "      <td>0.041827</td>\n",
       "      <td>0.490389</td>\n",
       "      <td>0.221963</td>\n",
       "      <td>0.109614</td>\n",
       "      <td>0.139203</td>\n",
       "      <td>3.224169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.075000</td>\n",
       "      <td>0.055000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.002000</td>\n",
       "      <td>0.001000</td>\n",
       "      <td>0.000500</td>\n",
       "      <td>0.001500</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.450000</td>\n",
       "      <td>0.350000</td>\n",
       "      <td>0.115000</td>\n",
       "      <td>0.441500</td>\n",
       "      <td>0.186000</td>\n",
       "      <td>0.093500</td>\n",
       "      <td>0.130000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.545000</td>\n",
       "      <td>0.425000</td>\n",
       "      <td>0.140000</td>\n",
       "      <td>0.799500</td>\n",
       "      <td>0.336000</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.234000</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.615000</td>\n",
       "      <td>0.480000</td>\n",
       "      <td>0.165000</td>\n",
       "      <td>1.153000</td>\n",
       "      <td>0.502000</td>\n",
       "      <td>0.253000</td>\n",
       "      <td>0.329000</td>\n",
       "      <td>11.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.815000</td>\n",
       "      <td>0.650000</td>\n",
       "      <td>1.130000</td>\n",
       "      <td>2.825500</td>\n",
       "      <td>1.488000</td>\n",
       "      <td>0.760000</td>\n",
       "      <td>1.005000</td>\n",
       "      <td>29.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                性别           长度           直径           高度         整体重量  \\\n",
       "count  4177.000000  4177.000000  4177.000000  4177.000000  4177.000000   \n",
       "mean      0.052909     0.523992     0.407881     0.139516     0.828742   \n",
       "std       0.822240     0.120093     0.099240     0.041827     0.490389   \n",
       "min      -1.000000     0.075000     0.055000     0.000000     0.002000   \n",
       "25%      -1.000000     0.450000     0.350000     0.115000     0.441500   \n",
       "50%       0.000000     0.545000     0.425000     0.140000     0.799500   \n",
       "75%       1.000000     0.615000     0.480000     0.165000     1.153000   \n",
       "max       1.000000     0.815000     0.650000     1.130000     2.825500   \n",
       "\n",
       "               肉重量         内脏重量           壳重           年龄  \n",
       "count  4177.000000  4177.000000  4177.000000  4177.000000  \n",
       "mean      0.359367     0.180594     0.238831     9.933684  \n",
       "std       0.221963     0.109614     0.139203     3.224169  \n",
       "min       0.001000     0.000500     0.001500     1.000000  \n",
       "25%       0.186000     0.093500     0.130000     8.000000  \n",
       "50%       0.336000     0.171000     0.234000     9.000000  \n",
       "75%       0.502000     0.253000     0.329000    11.000000  \n",
       "max       1.488000     0.760000     1.005000    29.000000  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:33:18.785128Z",
     "start_time": "2024-03-04T08:33:18.771750Z"
    }
   },
   "outputs": [],
   "source": [
    "# 提取特征和标签\n",
    "X = data.iloc[:,:-1]\n",
    "y = data.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:33:20.007313Z",
     "start_time": "2024-03-04T08:33:19.989712Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4177, 8)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:33:29.758752Z",
     "start_time": "2024-03-04T08:33:29.733073Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.1500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.0700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.2100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.1550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.0550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4172</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.450</td>\n",
       "      <td>0.165</td>\n",
       "      <td>0.8870</td>\n",
       "      <td>0.3700</td>\n",
       "      <td>0.2390</td>\n",
       "      <td>0.2490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4173</th>\n",
       "      <td>1</td>\n",
       "      <td>0.590</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.9660</td>\n",
       "      <td>0.4390</td>\n",
       "      <td>0.2145</td>\n",
       "      <td>0.2605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4174</th>\n",
       "      <td>1</td>\n",
       "      <td>0.600</td>\n",
       "      <td>0.475</td>\n",
       "      <td>0.205</td>\n",
       "      <td>1.1760</td>\n",
       "      <td>0.5255</td>\n",
       "      <td>0.2875</td>\n",
       "      <td>0.3080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4175</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.625</td>\n",
       "      <td>0.485</td>\n",
       "      <td>0.150</td>\n",
       "      <td>1.0945</td>\n",
       "      <td>0.5310</td>\n",
       "      <td>0.2610</td>\n",
       "      <td>0.2960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4176</th>\n",
       "      <td>1</td>\n",
       "      <td>0.710</td>\n",
       "      <td>0.555</td>\n",
       "      <td>0.195</td>\n",
       "      <td>1.9485</td>\n",
       "      <td>0.9455</td>\n",
       "      <td>0.3765</td>\n",
       "      <td>0.4950</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4177 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      性别     长度     直径     高度    整体重量     肉重量    内脏重量      壳重\n",
       "0      1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.1500\n",
       "1      1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.0700\n",
       "2     -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.2100\n",
       "3      1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.1550\n",
       "4      0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.0550\n",
       "...   ..    ...    ...    ...     ...     ...     ...     ...\n",
       "4172  -1  0.565  0.450  0.165  0.8870  0.3700  0.2390  0.2490\n",
       "4173   1  0.590  0.440  0.135  0.9660  0.4390  0.2145  0.2605\n",
       "4174   1  0.600  0.475  0.205  1.1760  0.5255  0.2875  0.3080\n",
       "4175  -1  0.625  0.485  0.150  1.0945  0.5310  0.2610  0.2960\n",
       "4176   1  0.710  0.555  0.195  1.9485  0.9455  0.3765  0.4950\n",
       "\n",
       "[4177 rows x 8 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:33:36.582832Z",
     "start_time": "2024-03-04T08:33:36.573055Z"
    }
   },
   "outputs": [],
   "source": [
    "# 切分训练集和测试集\n",
    "Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:33:49.446502Z",
     "start_time": "2024-03-04T08:33:49.436148Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2923, 8)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtrain.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1254, 8)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xtest.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:34:51.091668Z",
     "start_time": "2024-03-04T08:34:51.061735Z"
    }
   },
   "outputs": [],
   "source": [
    "# 建立线性回归模型\n",
    "# reg2 = LinearRegression().fit(Xtrain, Ytrain)\n",
    "reg2 = LinearRegression()\n",
    "reg2 = reg2.fit(Xtrain, Ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:34:56.790217Z",
     "start_time": "2024-03-04T08:34:56.772682Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([13.11640829,  9.65691091, 10.35350972, ...,  9.00574144,\n",
       "       18.83125845, 11.76923257])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat2 = reg2.predict(Xtest)\n",
    "yhat2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-04T08:35:23.830899Z",
     "start_time": "2024-03-04T08:35:23.821955Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0.08336487,  -1.19138201,  13.46629013,   8.1220863 ,\n",
       "         9.26547724, -20.45424715,  -9.7942646 ,   9.45690096])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看模型相关属性\n",
    "reg2.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('性别', 0.08336487137622806),\n",
       " ('长度', -1.1913820060178555),\n",
       " ('直径', 13.466290126335915),\n",
       " ('高度', 8.122086299064794),\n",
       " ('整体重量', 9.265477237852636),\n",
       " ('肉重量', -20.45424714682015),\n",
       " ('内脏重量', -9.794264604252113),\n",
       " ('壳重', 9.456900963011188)]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[*zip(X.columns,reg2.coef_)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0931295344110357"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg2.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5140139913856603"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg2.score(Xtest,Ytest) #R2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5327839192584529"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg2.score(Xtrain,Ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-5.432885182493395"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 计算MSE\n",
    "cross_val_score(reg2,X,y,cv=5,scoring=\"neg_mean_squared_error\").mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7 线性回归的进阶算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们在讲解线性回归原理的时候说过，回归系数的求解使用的是最小二乘法。但是这种方法的使用是有前提的，那就是：**特征之间不存在多重共线性，即特征之间完全独立**。多重共线性如果存在，则线性回归就无法使用最小二乘法来进行求解。在现实中特征之间完全独立的情况其实非常少，因为大部分数据统计手段或者收集者并不考虑统计学或者机器学习建模时的需求，现实数据多多少少都会存在一些多重共线性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "那么，为什么sklearn还是可以直接处理现实数据呢？（哪怕特征之间包含有多重共线性）那是因为，sklearn在封装线性回归算法的时候，在求解回归系数上稍稍做了修改，采用了取巧的方法来求解回归系数——SVD分解法，这是另一种求解回归系数的方法，它可以直接处理含有多重共线性的数据，代价就是损失一部分精确度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实，sklearn中还集成了一些进阶回归算法，在原有的线性回归算法基础上进行修改，使其能够容忍特征列存在多重共线性的情况，并且能够顺利建模，且尽可能的保证SSE取得最小值。比如：岭回归，Lasso，弹性网。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.1 岭回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在线性模型之中，除了线性回归之外，最知名的就是岭回归与Lasso了。这两个算法非常神秘，他们的原理和应用都不像其他算法那样高调，学习资料也很少。这可能是因为这两个算法**不是为了提升模型表现，而是为了修复漏洞**而设计的（实际上，我们使用岭回归或者Lasso，模型的效果往往会下降一些，因为我们删除了一小部分信息）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "岭回归在多元线性回归的损失函数上加上了正则项，表达为回归系数$w$的L2范式（即系数$w$的平方项）乘以正则化系数$\\alpha$。所以岭回归的损失函数的完整表达式写作："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\min\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\alpha {||\\boldsymbol{w}||_2}^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个操作看起来简单，其实带来了巨大的变化。正则化系数$\\alpha$能够避免多重共线性的影响，**$\\alpha$值越大，模型越不易受到共线性的影响**。但是$\\alpha$挤占了$w$中由原始的特征矩阵贡献的空间，因此$\\alpha$如果太大，也会导致$w$的估计出现较大的偏移，无法正确拟合数据的真实面貌。我们在使用中，需要找出$\\alpha$让模型效果变好的最佳取值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在sklearn中，岭回归由线性模型库中的Ridge类来调用："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*class* `sklearn.linear_model.Ridge`(*alpha=1.0*, *fit_intercept=True*, *normalize='deprecated'*, *copy_X=True*, *max_iter=None*, *tol=0.001*, *solver=’auto’*, *positive=False*，*random_state=None*)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和线性回归相比，岭回归的参数稍微多了那么一点点，但是真正核心的参数就是我们的**正则项的系数$\\alpha$**，其他的参数是当我们希望使用最小二乘法之外的求解方法求解岭回归的时候才需要的，通常我们完全不会去触碰这些参数。所以大家只需要了解$\\alpha$的用法就可以了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们选择上面使用鲍鱼数据集来验证岭回归这个算法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T02:42:33.885696Z",
     "start_time": "2024-03-06T02:42:25.825391Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import Ridge, LinearRegression, Lasso\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T02:42:33.947322Z",
     "start_time": "2024-03-06T02:42:33.890449Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "      <th>年龄</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   性别     长度     直径     高度    整体重量     肉重量    内脏重量     壳重  年龄\n",
       "0   1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1   1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2  -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3   1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4   0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入数据集\n",
    "data = pd.read_table('abalone.txt',header=None)\n",
    "data.columns=['性别','长度','直径','高度','整体重量','肉重量','内脏重量','壳重','年龄']\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T02:43:04.526988Z",
     "start_time": "2024-03-06T02:43:04.523909Z"
    }
   },
   "outputs": [],
   "source": [
    "# 提取特征和标签\n",
    "X = data.iloc[:,:-1]\n",
    "y = data.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T02:43:08.932721Z",
     "start_time": "2024-03-06T02:43:08.916610Z"
    }
   },
   "outputs": [],
   "source": [
    "# 切分训练集和测试集\n",
    "Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([13.05116455,  9.61888766, 10.69904254, ...,  9.50371951,\n",
       "       18.31028014, 11.77528163])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 建立岭回归模型\n",
    "reg3 = Ridge().fit(Xtrain, Ytrain)\n",
    "yhat3 = reg3.predict(Xtest)\n",
    "yhat3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5090428775885405"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R2 = reg3.score(Xtest,Ytest)\n",
    "R2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmP0lEQVR4nO3de3xU9Z3/8deHQLhfBVwgKAHihcrVyEWYQdwW0bKi3fVutVbX4upa608r2lblYX+7P3d5VB4WlB+td22RavFnV4uopQWUCgERkXsFMWI1IKDINeTz++NMyCSZkElIMjNn3s/HYx4z5zbznRN4nzOfc873mLsjIiLh1SzVDRARkcaloBcRCTkFvYhIyCnoRURCTkEvIhJyCnoRkZBT0IuIhJyCXrKOmW01s0Nm1rXK+FVm5mbWJ0VNE2kUCnrJVluAK8oHzGwg0Dp1zRFpPAp6yVbPANfEDV8LPF0+YGYtzWyamW0zs8/MbJaZtY5N62xm/2NmJWa2K/Y6L27ZP5vZA2b2lpl9ZWYLqv56EGlKCnrJVn8FOpjZ6WaWA1wGPBs3/UHgFGAI0B/oBdwbm9YMeAI4GTgJ2A/MqPL+VwLXAd2BXOCORvkWIklQ0Es2K9+r/xawHvgkNt6AfwV+5O5fuPtXwH8AlwO4+053f9Hd98Wm/W9gbJX3fsLdN7r7fmAuwQZDJCWap7oBIin0DLAIyCeubAN0A9oAK8ysfJwBOQBm1gZ4CJgAdI5Nb29mOe5+JDb897j32we0a4wvIJIM7dFL1nL3jwgOyl4A/D5u0g6Ccsw33L1T7NHR3cvD+n8BpwIj3L0DEI2NN0TSkIJest31wLnu/nXcuDLgV8BDZtYdwMx6mdl5sentCTYEu82sC3BfUzZYpK4U9JLV3P1v7l6UYNJdwGbgr2b2JfAGwV48wHSCUzF3EBzUnd8ETRWpN9ONR0REwk179CIiIaegFxEJOQW9iEjIKehFREIuLS+Y6tq1q/fp0yfVzRARyRgrVqzY4e7dEk1Ly6Dv06cPRUWJzngTEZFEzOyjmqapdCMiEnIKehGRkFPQi4iEnIJeRCTkFPQiIiGnoBcRCTkFvYhIyIUn6A8ehP/+b3j99VS3REQkrYQn6HNz4b/+C557LtUtERFJK0kFvZlNMLMNZrbZzKYcY76zzOyImf1LXZc9bmYQicCiRY32ESIimajWoDezHGAmcD4wALjCzAbUMN+DwGt1XbbBRKOwZQsUFzfaR4iIZJpk9uiHA5vd/UN3PwTMASYlmO/fgReBz+uxbMOIRILnxYsb7SNERDJNMp2a9QI+jhsuBkbEz2BmvYCLgXOBs+qybNx73AjcCHDSSScl0awEyp6C+3Jg9x3wxv+t33uIiKRK5yFw5vQGf9tk9ugtwbiqN5qdDtzl7kfqsWww0n22uxe6e2G3bgl72qxds2bQoQPs3l2/5UVEQiiZPfpioHfccB6wvco8hcAcMwPoClxgZqVJLttwzpwOr3WHn/wEdrwIJ5zQaB8lIpIpktmjXw4UmFm+meUClwMvx8/g7vnu3sfd+wAvAP/m7i8ls2yDi0aD5yVLGvVjREQyRa1B7+6lwC0EZ9OsA+a6+wdmNtnMJtdn2eNv9jGcdRa0bKkDsiIiMeaesGSeUoWFhX5cd5gaOxb274dlyxquUSIiaczMVrh7YaJp4bkyNl4kAitXwt69qW6JiEjKhTPoo1E4cgSWLk11S0REUi6cQT9qVHCqper0IiIhDfr27WHYMPV7IyJCWIMegjr9O+8E3ReLiGSx8AZ9NAoHDsDxnL0jIhIC4Q36MWOCZ9XpRSTLhTfou3aFAQNUpxeRrBfeoIegTv/WW8GpliIiWSrcQR+NwpdfwurVqW6JiEjKhDvodSMSEZGQB33v3tCnj+r0IpLVwh30EOzVL14Madh5m4hIUwh/0Eej8PnnsHFjqlsiIpIS4Q961elFJMuFP+hPOQW6d1edXkSyVviD3qyiTi8ikoXCH/QQ1Om3boWPP051S0REmlx2BL3q9CKSxbIj6AcNgg4dVKcXkayUHUGfkwOjR2uPXkSyUlJBb2YTzGyDmW02sykJpk8ys9VmtsrMisxsTNy0rWb2fvm0hmx8nUSjsHYt7NiRsiaIiKRCrUFvZjnATOB8YABwhZkNqDLbm8Bgdx8CfB/4dZXp49x9iLsXHn+T66m8Tr9kScqaICKSCsns0Q8HNrv7h+5+CJgDTIqfwd33uh/tY6AtkH79DRQWQqtWqtOLSNZJJuh7AfHnJRbHxlViZheb2XrgFYK9+nIOLDCzFWZ2Y00fYmY3xso+RSUlJcm1vi5atoQRI1SnF5Gsk0zQW4Jx1fbY3X2eu58GXAQ8EDdptLsPIyj93Gxm0UQf4u6z3b3Q3Qu7deuWRLPqIRqFlSvhq68a5/1FRNJQMkFfDPSOG84Dttc0s7svAvqZWdfY8PbY8+fAPIJSUGpEIlBWBkuXpqwJIiJNLZmgXw4UmFm+meUClwMvx89gZv3NzGKvhwG5wE4za2tm7WPj2wLjgTUN+QXqZNSo4FRL1elFJIs0r20Gdy81s1uA14Ac4HF3/8DMJsemzwL+GbjGzA4D+4HL3N3N7ERgXmwb0Bz4jbvPb6TvUrt27WDYMNXpRSSrmKfhDTkKCwu9qKiRTrm/4w6YMQP27AkO0IqIhICZrajpFPbsuDI2XiQCBw/C8uWpbomISJPIvqAfE7toV3V6EckS2Rf0J5wA3/iG6vQikjWyL+ghOJ/+rbfgyJFUt0REpNFlZ9BHIsFFU++9l+qWiIg0uuwNelCdXkSyQnYGfV4e5OerTi8iWSE7gx6COv3ixZCG1xGIiDSk7A36SARKSmDDhlS3RESkUWVv0EdjnWiqTi8iIZe9Qd+/P5x4our0IhJ62Rv0ZsFevfboRSTksjfoIajTb9sGH32U6paIiDSa7A768jq9yjciEmLZHfRnnAEdOyroRSTUsjvoc3KC3ixVpxeREMvuoIegTr9+PXz+eapbIiLSKBT05XX6JUtS2w4RkUaioD/zTGjdWnV6EQktBX1uLowcqTq9iISWgh6COv2qVfDll6luiYhIg0sq6M1sgpltMLPNZjYlwfRJZrbazFaZWZGZjUl22bQQjUJZGbz9dqpbIiLS4GoNejPLAWYC5wMDgCvMbECV2d4EBrv7EOD7wK/rsGzqjRwJzZurTi8ioZTMHv1wYLO7f+juh4A5wKT4Gdx9r/vRjt3bAp7ssmmhbdvgoKzq9CISQskEfS/g47jh4ti4SszsYjNbD7xCsFef9LKx5W+MlX2KSkpKkml7w4pEYNkyOHCg6T9bRKQRJRP0lmBctdsyufs8dz8NuAh4oC7Lxpaf7e6F7l7YrVu3JJrVwKJROHQoCHsRkRBJJuiLgd5xw3nA9ppmdvdFQD8z61rXZVNq9OjgWeUbEQmZZIJ+OVBgZvlmlgtcDrwcP4OZ9Tczi70eBuQCO5NZNm106QIDB+qArIiETvPaZnD3UjO7BXgNyAEed/cPzGxybPos4J+Ba8zsMLAfuCx2cDbhso30XY5fJAJPPw2lpcFZOCIiIWAVJ8ukj8LCQi8qKmr6D37+ebj8cli+HAoLm/7zRUTqycxWuHvC4NKVsfEikeBZdXoRCREFfbyePaFfP9XpRSRUFPRVRSJB0JeVpbolIiINQkFfVTQKO3cGNyMREQkBBX1VqtOLSMgo6Kvq1w969FCdXkRCQ0FflVmwV79oEaThqaciInWloE8kGoXiYvjoo1S3RETkuCnoE1GdXkRCREGfyBlnQKdOqtOLSCgo6BNp1gzGjNEevYiEgoK+JtEobNwIn32W6paIiBwXBX1Nyuv0Kt+ISIZT0Ndk2DBo00ZBLyIZT0Ffk9xcGDlSdXoRyXgK+mOJRuG992DPnlS3RESk3hT0xxKJBFfHvvVWqlsiIlJvCvpjGTkyuKWg6vQiksEU9MfSpk1wS0HV6UUkgynoaxONBveQ3b8/1S0REakXBX1tIhE4fBjeeSfVLRERqZekgt7MJpjZBjPbbGZTEky/ysxWxx5vm9nguGlbzex9M1tlZkUN2fgmMXp00HWx6vQikqGa1zaDmeUAM4FvAcXAcjN72d3Xxs22BRjr7rvM7HxgNjAibvo4d9/RgO1uOp07w8CBqtOLSMZKZo9+OLDZ3T9090PAHGBS/Azu/ra774oN/hXIa9hmplg0CkuXBiUcEZEMk0zQ9wI+jhsujo2ryfXAH+OGHVhgZivM7MaaFjKzG82syMyKSkpKkmhWE4pE4Ouv4d13U90SEZE6SyboLcG4hPfYM7NxBEF/V9zo0e4+DDgfuNnMoomWdffZ7l7o7oXdunVLollNSB2ciUgGSyboi4HeccN5wPaqM5nZIODXwCR331k+3t23x54/B+YRlIIyS48e0L+/6vQikpGSCfrlQIGZ5ZtZLnA58HL8DGZ2EvB74LvuvjFufFsza1/+GhgPrGmoxjepaBSWLIGyslS3RESkTmoNencvBW4BXgPWAXPd/QMzm2xmk2Oz3QucADxS5TTKE4ElZvYesAx4xd3nN/i3aAqRCHzxBaxdW/u8IiJppNbTKwHc/VXg1SrjZsW9vgG4IcFyHwKDq47PSNHYoYXFi4N7yoqIZAhdGZus/Hzo2VN1ehHJOAr6ZJkFe/WLFwddF4uIZAgFfV1EIvDJJ7BlS6pbIiKSNAV9XcTX6UVEMoSCvi4GDAj6vlGdXkQyiIK+Lpo1C8o32qMXkQyioK+rSAQ2bYK//z3VLRERSYqCvq5UpxeRDKOgr6uhQ4N7yapOLyIZQkFfVy1awNlna49eRDKGgr4+IhFYvRp27051S0REaqWgr49oNLg69q23Ut0SEZFaKejrY8SIoISjOr2IZAAFfX20bg1nnaU6vYhkBAV9fUUisHw57NuX6paIiByTgr6+olEoLYV33kl1S0REjklBX19nnx10Xaw6vYikOQV9fXXqBIMHq04vImlPQX88IhFYuhQOH051S0REaqSgPx7RaHAwduXKVLdERKRGCvrjEYkEz6rTi0gaSyrozWyCmW0ws81mNiXB9KvMbHXs8baZDU522Yx24olwyimq04tIWqs16M0sB5gJnA8MAK4wswFVZtsCjHX3QcADwOw6LJvZIhFYsgTKylLdEhGRhJLZox8ObHb3D939EDAHmBQ/g7u/7e67YoN/BfKSXTbjRaOwaxd88EGqWyIiklAyQd8L+DhuuDg2ribXA3+s67JmdqOZFZlZUUlJSRLNShOq04tImksm6C3BOE84o9k4gqC/q67Luvtsdy9098Ju3bol0aw00acP5OWpTi8iaSuZoC8GescN5wHbq85kZoOAXwOT3H1nXZbNaGbBXv2iRUHXxSIiaSaZoF8OFJhZvpnlApcDL8fPYGYnAb8HvuvuG+uybChEo/Dpp/Dhh6luiYhINc1rm8HdS83sFuA1IAd43N0/MLPJsemzgHuBE4BHzAygNFaGSbhsI32X1Imv0/frl9q2iIhUYZ6G5YbCwkIvKipKdTOSV1YG3bvDhRfC44+nujUikoXMbIW7FyaapitjG0KzZjBmjM68EZG0pKBvKNEo/O1vsD1cx5pFJPMp6BtKeZ1ep1mKSJpR0DeUoUOhbVsFvYikHQV9Q2nePLjrlOr0IpJmFPQNKRqFNWvgiy9S3RIRkaMU9A0pEgmujn3rrVS3RETkKAV9Qxo+HHJzVacXkbSioG9IrVvDWWepTi8iaUVB39CiUVixAr7+OtUtEREBFPQNLxKB0lKYMSN4FhFJMQV9QzvnHBg9GqZMgQED4Le/1W0GRSSlFPQNrXXr4GDsSy9Bq1Zw5ZUweDDMm6f+6kUkJRT0jcEMJk2CVatgzhw4dAi+853gQO0f/6jAF5EmpaBvTM2awWWXBTcOf+IJ2LkTLrggqOP/+c+pbp2IZAkFfVNo3hy+9z3YsAEefRS2bIFx4+Cb34S//jXVrRORkFPQN6XcXJg8GTZvhocegtWrYdQomDgR3n031a0TkZBS0KdC69Zw223BPWb/4z+CLhOGDYNLLoG1a1PdOhEJGQV9KrVrB3ffHZRy7r0X5s+HM86A73432OsXEWkACvp00KkTTJ0aBP6dd8KLL8Jpp8G//its25bq1olIhlPQp5OuXeHBB4NbEv7bv8HTT0NBAdx6K3z6aapbJyIZKqmgN7MJZrbBzDab2ZQE008zs6VmdtDM7qgybauZvW9mq8ysqKEaHmo9esDDD8OmTXDttfDII9CvH/z4x7BjR6pbJyIZptagN7McYCZwPjAAuMLMBlSZ7QvgVmBaDW8zzt2HuHvh8TQ265x0EsyeDevXw7/8C0ybBvn5QT1/9+5Ut05EMkQye/TDgc3u/qG7HwLmAJPiZ3D3z919OXC4Edoo/fsHZZw1a2DCBHjgAejbF/7zP2Hv3lS3TkTSXDJB3wv4OG64ODYuWQ4sMLMVZnZjTTOZ2Y1mVmRmRSUlJXV4+ywyYAD87newcmXQcdo99wSB/9BDcOBAqlsnImkqmaC3BOPq0lnLaHcfRlD6udnMoolmcvfZ7l7o7oXdunWrw9tnoaFD4Q9/gKVLgw7Tbr892OufNSvoV0dEJE4yQV8M9I4bzgO2J/sB7r499vw5MI+gFCQNYeRIeP11WLgQ+vSBm26CU0+FJ59UX/giclQyQb8cKDCzfDPLBS4HXk7mzc2srZm1L38NjAfW1LexUoNzzgm6Rv7jH+GEE+C66+Ab34Bnn9VBWxGpPejdvRS4BXgNWAfMdfcPzGyymU0GMLN/MLNi4Hbgp2ZWbGYdgBOBJWb2HrAMeMXd5zfWl8lqZsGB2uXLg77vc3ODK2y7dIFBg+Dmm4OboHz8ce3vJSKhYp6GfaMXFhZ6UZFOuT8uZWXBTcoXL4YlS+DttyvO0DnppKCr5DFjgseAAUGXyiKSscxsRU2nsDdv6sZIE2nWLCjpnHNOMFxaCu+/XxH8b74Jzz0XTOvcOTiLpzz4CwuhZctUtVxEGpj26LOVe9C3zpIlFeG/fn0wrWXL4G5Y5Xv9Z58d9McjImnrWHv0CnqpUFISlHjKw3/FiuCXgFnQq+aYMRXh37t37e8nIk1GQS/1s28fLFtWEfxV6/zxwa86v0hKqUYv9dOmTeI6f3nwL1wIv/lNMK1Tp6DOXx78qvOLpA3t0Uv9xdf5y8O/ap2/vMY/aFDwK8ASXWgtIsdLpRtpOjXV+QE6dAhq/YMGwcCBFQ8d6BU5bgp6SZ19+2DVqqDks3p18Pz++5Wv2O3du3LwDxwY3GErNzdVrRbJOKrRS+q0aROUbs4+u2KcO3zySfXwf/11OBzr6bp58yDs48N/0KBgo6Dyj0idKOil6ZlBXl7wOP/8ivGHD8OGDRXB//77QRnot7+tmKdjx6D8Ux78AwcGwyr/iNQoY0o3hw8fpri4mAPqdz2UWrVqRV5eHi1atKg+cc+e4KYr8RuAmso/8fX/U09V+UeyRihKN8XFxbRv354+ffpg+ukeKu7Ozp07KS4uJj8/v/oMHTsGp26OHh2/EBQXVw7+1asrl39atKhc/jnjDDj99KBL55ycJvluIukgY4L+wIEDCvmQMjNOOOEE6nRnMbNgL753b7jggorxhw7Bxo2Vw3/Jkorz/SE49fPUU4PQj3+ccorO/ZdQypigBxTyIdZgf9vc3GDP/Ywz4IorKsbv2QNr18K6dRWP5cth7tzg1wEEV/b27Vt9A3DaacGvCpEMlVFBL1JvHTvCqFHBI97+/cEvgPgNwLp18NprlW/L2LNn9Q3A6afDiSfqLCBJewr6OsjJyWHgwIGUlpaSn5/PM888Q6dOndi+fTu33norL7zwQrVlzjnnHKZNm0ZhYcJjJJJqrVsH990dPLjy+NLS4KrfqhuAp56Cr76qmK9Tp8QbgJNP1nEASRsK+jpo3bo1q1atAuDaa69l5syZ/OQnP6Fnz54JQ14yWPPmUFAQPC68sGK8O2zfXn0D8Mor8PjjFfO1apX4OEBBgY4DSJPLzKC/7bbgasuGNGQITJ+e9OyjRo1i9erVAGzdupWJEyeyZs0a9u/fz3XXXcfatWs5/fTT2b9//9FlHnvsMR588EF69uxJQUEBLVu2ZMaMGZSUlDB58mS2bdsGwPTp0xkdf4aJpA8z6NUreHzzm5Wn7dpVfQPwzjvw/PPVjwOcempQ+49/7tZNZSBpFJkZ9Cl25MgR3nzzTa6//vpq0x599FHatGnD6tWrWb16NcOGDQNg+/btPPDAA6xcuZL27dtz7rnnMjhWLvjhD3/Ij370I8aMGcO2bds477zzWLduXZN+J2kAnTtXvwoYgm4g4o8DrF8fXBj25psQf11I586JNwD9+ul6ADkumRn0ddjzbkj79+9nyJAhbN26lTPPPJNvfetb1eZZtGgRt956KwCDBg1i0KBBACxbtoyxY8fSpUsXAC655BI2btwIwBtvvMHatWuPvseXX37JV199Rfv27Rv7K0lTaNMm+MU4ZEjl8WVlsG1bRfCXPy9YAE8+WTFfTk7NvwK6dtWvAKlVZgZ9ipTX6Pfs2cPEiROZOXPm0VCPl+hUwWNdgVxWVsbSpUtp3bp1g7ZX0lyzZsHFW336wIQJlad9+WXwK6DqRuD11+HgwYr5OneuHv6nnRb8Ckh0lbFkpaRuCWRmE8xsg5ltNrMpCaafZmZLzeygmd1Rl2UzUceOHXn44YeZNm0ah8uvwoyJRqM8F7vp9po1a47W8YcPH85f/vIXdu3aRWlpKS+++OLRZcaPH8+MGTOODq9q6OMPknk6dAhu3nL11fDAA/C73wUXf339NXz4Ibz6Kjz0EFx6aXBwd/58mDIFLr44OOjbunUQ+hdeCD/+MTz2GLz1FuzYkepvJilQ6x69meUAM4FvAcXAcjN72d3Xxs32BXArcFE9ls1IQ4cOZfDgwcyZM4dIJHJ0/E033cR1113HoEGDGDJkCMOHDwegV69e3HPPPYwYMYKePXsyYMAAOsYuwnn44Ye5+eabGTRoEKWlpUSjUWbNmpWS7yVpLicH8vODR3yHcBBcFLZhQ+VfAOvXV78moEuXir3/U06peO7fX2cEhVStnZqZ2Sjgfnc/LzZ8N4C7/2eCee8H9rr7tLouGy9Rp2br1q3j9NNPT+5bpam9e/fSrl07SktLufjii/n+97/PxRdfnOpmpY0w/I3T0pEj8NFH1ctAGzbA3/9eMV+zZsH5//EbgPLXvXrpnsBp7ng7NesFfBw3XAyMSPKzk17WzG4EbgQ46aSTknz7zHL//ffzxhtvcODAAcaPH89FF12U6iZJNig/mNu3b+V+gaDiWMDGjUHwlz8vXhyUicq1aRNcA1D1V8Cpp6p7iAyQTNAnOqSfbN/GSS/r7rOB2RDs0Sf5/hll2rRpqW6CSGXlxwKqXrldfmFY1Q3AypXw4ovBr4Ry3bsn3gD07avTQtNEMkFfDPSOG84Dtif5/sezrIikSvyFYePGVZ526FBwQDh+A7BxI/zhD8FB33LlxxMSbQR69NBpoU0omaBfDhSYWT7wCXA5cGWS7388y4pIOsrNDQ7mnnZa9Wm7dycuBf3pT0EHcuXatQtCP/5RUBA8625hDa7WoHf3UjO7BXgNyAEed/cPzGxybPosM/sHoAjoAJSZ2W3AAHf/MtGyjfRdRCTVOnWC4cODR7yysuA+wVU3AO+8E3QVXVZWMW+3bhWhX/5cflZQmzZN+nXCImNuJagzMsJPf+MsdfBg0FNo+S+BTZsqXm+vUunNy6sc/uWv8/Oz/nhAKG4lmA7atWvH3r17K42bNWsWbdq04ZprrmnUz+7Tpw/t27fHzOjcuTNPP/00J598cqN+ZrKaah1ISLVsWXMpaO9e2Ly58gZg0yZ44QXYubNivpyc4Arjqr8CCgqCu5BleZfR2qOvg0RB39jcHXenb9++FBUV0bVrV+677z62b9/Or371qwZ572Zpcn50OvyNJYN88UXlvf/41/GnhrZsGZR9EpWDQnTjmPDt0a+4DXatatj37DwEzpxe58Xuv/9+2rVrxx133ME555zDiBEjWLhwIbt37+axxx4jEolw5MgRpkyZwp///GcOHjzIzTffzA9+8AP27t3LpEmT2LVrF4cPH+bnP/85kyZNYuvWrZx//vmMGzeOpUuX8tJLL1X6zFGjRvHwww8D1NjFcUlJCVdeeSU7d+7krLPOYv78+axYsYK9e/dWe++5c+cyd+5cDh48yMUXX8zUqVP5+uuvufTSSykuLubIkSP87Gc/47LLLmPKlCm8/PLLNG/enPHjxzNt2rRK62DVqlVMnjyZffv20a9fPx5//HE6d+5c47oRqbcuXWDEiOARzz24ECzRBuDVVytfJVx+ULj83gPlG4SCglB1GJeZQZ/GSktLWbZsGa+++ipTp07ljTfe4LHHHqNjx44sX76cgwcPMnr0aMaPH0/v3r2ZN28eHTp0YMeOHYwcOZILYze52LBhA0888QSPPPJItc+YP3/+0YutaurieOrUqZx77rncfffdzJ8/n9mzZx9dPv69FyxYwKZNm1i2bBnuzoUXXsiiRYsoKSmhZ8+evPLKKwDs2bOHL774gnnz5rF+/XrMjN27d1dr2zXXXMMvf/lLxo4dy7333svUqVOZHuttNNG6EWlwZsHpmz16wNixlacdORL0GFr1l8Dy5UF/QvEHhTt2rAj+qs8ZthHIzKCvx553U/nOd74DwJlnnsnWrVsBWLBgAatXrz56F6o9e/awadMm8vLyuOeee1i0aBHNmjXjk08+4bPPPgPg5JNPZuTIkZXee9y4cXz22Wd0796dn//850DNXRwvWbKEefPmATBhwgQ6d+58dJ74916wYAELFixg6NChQNBNw6ZNm4hEItxxxx3cddddTJw4kUgkQmlpKa1ateKGG27g29/+NhMnTqzUvj179rB7927Gxv5zXXvttVxyySXHXDciTSq+r6Dx4ytPO3QItm4Ngn/z5ornZcuqnxmUaCNQ/joNNwKZGfRprGWsU6icnBxKS0uBoBb+y1/+kvPOO6/SvE8++SQlJSWsWLGCFi1a0KdPHw7EbkTRtm3bau+9cOFC2rZty/e+9z3uvfdefvGLX9TYxfGxjr3Ev7e7c/fdd/ODH/yg2nwrVqzg1Vdf5e6772b8+PHce++9LFu2jDfffJM5c+YwY8YM/vSnPyW5ZhKvG5G0kZtbUbuvKtFGYNOm2jcCVX8NpGgjoKBvAueddx6PPvoo5557Li1atGDjxo306tWLPXv20L17d1q0aMHChQv56KOPan2v1q1bM336dAYOHMhPf/rTo10c33nnnUDQxfGQIUMYM2YMc+fO5a677mLBggXs2rWrxrb97Gc/46qrrqJdu3Z88skntGjRgtLSUrp06cLVV19Nu3btePLJJ9m7dy/79u3jggsuYOTIkfTv37/Se3Xs2JHOnTuzePFiIpEIzzzzzNG9e5GMluxGIH5DUNNGIFEpqKAATjih0TYCCvo62LdvH3l5eUeHb7/99qSWu+GGG9i6dSvDhg3D3enWrRsvvfQSV111Ff/0T/9EYWEhQ4YM4bREp5cl0KNHD6644gpmzpxZYxfH9913H1dccQXPP/88Y8eOpUePHrRv377aWUPjx49n3bp1jBo1CgjOLHr22WfZvHkzd955J82aNaNFixY8+uijfPXVV0yaNIkDBw7g7jz00EPV2vbUU08dPRjbt29fnnjiiaS+k0jGqm0jsGVL5VLQsTYCgwbBX/7S4IGv0ytD6uDBg+Tk5NC8eXOWLl3KTTfdlPY3NNHfWLJKoo3AoUMQd+JEXYTv9Eqp1bZt27j00kspKysjNzf3uM+5F5EGlptb0ed/I1PQh1RBQQHvvvtuqpshImkgPS6JTFI6lpmkYehvK9J4MiboW7Vqxc6dOxUIIeTu7Ny5k1atWqW6KSKhlDGlm7y8PIqLiykpKUl1U6QRtGrVqtIZTSLScDIm6Fu0aEF+fn6qmyEiknEypnQjIiL1o6AXEQk5Bb2ISMil5ZWxZlYC1N7xS3VdgR0N3JxMpvVRmdZHBa2LysKwPk52926JJqRl0NeXmRXVdAlwNtL6qEzro4LWRWVhXx8q3YiIhJyCXkQk5MIW9PXr9i28tD4q0/qooHVRWajXR6hq9CIiUl3Y9uhFRKQKBb2ISMiFJujNbIKZbTCzzWY2JdXtaWxm1tvMFprZOjP7wMx+GBvfxcxeN7NNsefOccvcHVs/G8zsvJrfPXOZWY6ZvWtm/xMbztr1YWadzOwFM1sf+3cyKlvXh5n9KPb/ZI2Z/dbMWmXVunD3jH8AOcDfgL5ALvAeMCDV7Wrk79wDGBZ73R7YCAwA/guYEhs/BXgw9npAbL20BPJj6ysn1d+jEdbL7cBvgP+JDWft+gCeAm6Ivc4FOmXj+gB6AVuA1rHhucD3smldhGWPfjiw2d0/dPdDwBxgUorb1Kjc/VN3Xxl7/RWwjuAf9CSC/+DEni+KvZ4EzHH3g+6+BdhMsN5Cw8zygG8Dv44bnZXrw8w6AFHgMQB3P+Tuu8nS9UHQU29rM2sOtAG2k0XrIixB3wv4OG64ODYuK5hZH2Ao8A5wort/CsHGAOgemy0b1tF04MdAWdy4bF0ffYES4IlYKevXZtaWLFwf7v4JMA3YBnwK7HH3BWTRughL0FuCcVlx3qiZtQNeBG5z9y+PNWuCcaFZR2Y2Efjc3Vcku0iCcaFZHwR7sMOAR919KPA1QXmiJqFdH7Ha+ySCMkxPoK2ZXX2sRRKMy+h1EZagLwZ6xw3nEfw0CzUza0EQ8s+5++9joz8zsx6x6T2Az2Pjw76ORgMXmtlWgtLduWb2LNm7PoqBYnd/Jzb8AkHwZ+P6+Cawxd1L3P0w8HvgbLJoXYQl6JcDBWaWb2a5wOXAyyluU6MyMyOov65z91/ETXoZuDb2+lrg/8WNv9zMWppZPlAALGuq9jY2d7/b3fPcvQ/B3/9P7n412bs+/g58bGanxkb9I7CW7Fwf24CRZtYm9v/mHwmOaWXNusiYWwkei7uXmtktwGsEZ+A87u4fpLhZjW008F3gfTNbFRt3D/B/gLlmdj3BP/BLANz9AzObS/CfvRS42d2PNHmrm142r49/B56L7fx8CFxHsHOXVevD3d8xsxeAlQTf7V2CLg/akSXrQl0giIiEXFhKNyIiUgMFvYhIyCnoRURCTkEvIhJyCnoRkZBT0IuIhJyCXkQk5P4/gjOhq2aBIKcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#交叉验证下，与线性回归相比，岭回归的结果如何变化？\n",
    "alpharange = np.arange(1,1001,100)\n",
    "ridge_re = []\n",
    "linear_re = []\n",
    "for alpha in alpharange:\n",
    "    ridge = Ridge(alpha=alpha)\n",
    "    linear = LinearRegression()\n",
    "    ridge_r2 = cross_val_score(ridge,X,y,cv=5,scoring = \"r2\").mean()\n",
    "    linear_r2 = cross_val_score(linear,X,y,cv=5,scoring = \"r2\").mean()\n",
    "    ridge_re.append(ridge_r2)\n",
    "    linear_re.append(linear_r2)\n",
    "plt.plot(alpharange,ridge_re,color=\"red\",label=\"Ridge\")\n",
    "plt.plot(alpharange,linear_re,color=\"orange\",label=\"LinearRegression\")\n",
    "plt.title(\"Mean\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEICAYAAAC0+DhzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAx6UlEQVR4nO3de5zWc/7/8cezaSodKJl2O1FLlmQ6mHKsVukgrYq1siw2p8jXaVmy1mFZtEtaikRCDmmlZNmUtA5LaqL6dVTaMGU1QkrHqdfvj/dn6qqZmmummfnM4XW/3a7bXNfn+PpMzOt6n2VmOOecc4mqxB2Ac865sseTg3POuTw8OTjnnMvDk4Nzzrk8PDk455zLw5ODc865PDw5OOecy8OTg3MFkLRS0lZJh+yxfa4kk9QsptCcKzGeHJxLzn+B83I/SDoWOCC+cJwrWZ4cnEvOWODChM8XAc/mfpBUXdIDkr6Q9LWkkZIOiPbVk/RPSdmSvoveN0k499+S7pb0H0nrJU3ds5TiXGnz5OBccmYCB0o6WlIKcC7wXML+IcCRQBvgCKAxcHu0rwowBjgMOBTYBAzf4/q/AX4HNACqATeWyFM4lyRPDs4lL7f00A1YAqyKtgu4DLjezL41s/XAvUB/ADNba2YTzGxjtO8vQOc9rj3GzD41s03AeEKScS42VeMOwLlyZCzwLtCchColIA2oCcyRlLtNQAqApJrAQ0BPoF60v46kFDPbHn3+X8L1NgK1S+IBnEuWlxycS5KZfU5omO4FvJKw6xtCVdExZlY3eh1kZrl/4H8P/Bw43swOBDpF24VzZZQnB+cK5xKgi5n9mLBtB/AE8JCkBgCSGkvqEe2vQ0ge30s6GLijNAN2rig8OThXCGb2mZll5rPrZmA5MFPSD8BbhNICwDBCt9dvCA3bU0ohVOf2i3yxH+ecc3vykoNzzrk8PDk455zLw5ODc865PDw5OOecy6NCDII75JBDrFmzZnGH4Zxz5cqcOXO+MbO0/PZViOTQrFkzMjPz613onHNubyR9vrd9Xq3knHMuD08Ozjnn8vDk4JxzLo8K0ebgnCu6bdu2kZWVxebNm+MOxZWQGjVq0KRJE1JTU5M+x5ODc5VcVlYWderUoVmzZiRMOe4qCDNj7dq1ZGVl0bx586TP82ol5yq5zZs3U79+fU8MFZQk6tevX+iSoScH55wnhgquKP++Xq1U3m3ZAgsXwrx5sGoV1KsH9evveh1ySPhZsyb4HwDnXJI8OZQna9aEJDBvHsydG34uWQI5OQWfW7367ski9/Xzn0OXLnDssVDFC5IuHikpKRx77LHk5OTQvHlzxo4dS926dVm9ejXXXHMNL7/8cp5zfvGLX/DAAw+QkZERQ8QVnyeHsmr5csjM3JUE5s2Dr77atb9xY2jTBs48E1q3Du8POwy+/x7Wrt399c03ebctXAjZ2WEfhERx6qkhUXTpAkce6SUNV2oOOOAA5s6dC8BFF13EiBEj+OMf/0ijRo3yTQyu5HlyKGuysuAPf4AXXwyfU1OhZUvo1m1XEkhPDyWA/PzkJ+GVrC+/hBkz4O23wyv3f8RGjXYlii5dQuJxrhSceOKJzJ8/H4CVK1fSu3dvFixYwKZNm/jd737HokWLOProo9m0adPOc0aPHs2QIUNo1KgRLVq0oHr16gwfPpzs7GwGDhzIF198AcCwYcM4+eSTY3mu8saTQ1mxeTMMHQp/+Qts3w5//COccw4cfTRUq1Zy923aFC68MLzM4LPPQpKYMQOmToXnngvHNW++K1H07g0HHlhyMbn4XHddKK0WpzZtYNiwpA7dvn0706dP55JLLsmz77HHHqNmzZrMnz+f+fPn065dOwBWr17N3Xffzccff0ydOnXo0qULrVu3BuDaa6/l+uuv55RTTuGLL76gR48eLF68uLierELz5BA3M3jtNbj+elixAvr1gwcfDH+MS5sERxwRXpdfHmJbtGhXqWLCBBg9OjRun3suXHYZnHCCVz+5/bZp0ybatGnDypUrOe644+jWrVueY959912uueYaANLT00lPTwdg1qxZdO7cmYMPPhiAc845h08//RSAt956i0WLFu28xg8//MD69eupU6dOST9SuefJIU5LloRvam++GUoI06bBaafFHdUuEhxzTHj93/+FEs1HH8GYMaHaa8yYsO/SS+G3vw3tFq58S/IbfnHLbXNYt24dvXv3ZsSIETsTQaL8umSa2V6vu2PHDj788EMOOOCAYo23MvDuKXFYtw5+//vQQ2jmzPA/5Lx5ZSsx5CclBU46CZ54IjSOjxoFtWqFUk+jRvCb34QSxo4dcUfqyqmDDjqIhx9+mAceeIBt27bttq9Tp048//zzACxYsGBnu0SHDh145513+O6778jJyWHChAk7z+nevTvDhw/f+XlucVeZVWCeHErTjh3h2/aRR8JDD8HFF8Onn8K114aG5/KkTp1QrfTRRyGxXXEF/Otf0LVreL777tu9d5VzSWrbti2tW7dm3Lhxu22/8sor2bBhA+np6fz1r3+lQ4cOADRu3Jhbb72V448/ntNOO42WLVty0EEHAfDwww+TmZlJeno6LVu2ZOTIkaX+POWWmZX713HHHWdl3syZZu3bm4HZiSeaZWbGHVHx27jR7LnnzDp3Ds+ZkmLWt6/ZG2+Y7dgRd3RuLxYtWhR3CPtt/fr1Zma2bds26927t73yyisxR1T25PfvDGTaXv6uesmhpJmFksEJJ4RuqmPHwn/+A8cdF3dkxe+AA+D88+Hf/4alS0PV2QcfQK9e4XknTw6/D+eK2Z133kmbNm1o1aoVzZs3p2/fvnGHVP7tLWskvoCewFJgOXDLPo5rD2wHflXQucCdwCpgbvTqlbBvcHT8UqBHQfGV6ZLD0KHhW/RVV5n98EPc0ZS+rVvNxowxO/zw8Hto29Zs0iQvSZQhFaHk4ApW7CUHSSnACOB0oCVwnqSWezluCPBmIc59yMzaRK83onNaAv2BY6LE8mh0nfJnxgy46abQPXX48FBPX9mkpoa2lSVL4OmnYf166Ns3lCQmTfKShHNlVDLVSh2A5Wa2wsy2AuOAPvkc93/ABGBNEc5N1AcYZ2ZbzOy/hBJEhyTiLFu+/DKMBWjRIvxRrOxjAapWhYsugsWL4ZlnQpLo1w/atfMk4VwZlExyaAx8mfA5K9q2k6TGQD9gz64ABZ17taT5kp6SVC/Z+5V5mzfDWWeFn5Mm+WjiRFWrhtHYixfDs8/Cjz+GJNG2LUyc6N1gnSsjkkkO+X3l3fNr3jDgZjPbXohzHwMOB9oAXwEPFuJ+SLpcUqakzOzs7Pwjj4MZDBoUJs0bOzbMeuryqlo1DJxbtCgkiY0bQ0Jt186ThHNlQDLJIQtomvC5CbB6j2MygHGSVgK/IrQT9N3XuWb2tZltN7MdwBPsqjpK5n6Y2SgzyzCzjLS0tCQeo5Q8/jg89RTcdhv0KagGze2WJMaO3ZUkOnQIvbpcpVC7du0820aOHMmzzz5b4vdu1qwZxx57LOnp6XTu3JnPP/+8xO+ZrNL6HeRrby3VtqvnUFVgBdAcqAbMA47Zx/FPE/VW2te5QMOEc64ntDNAaIieB1SPzlsBpOwrxjLTW+mDD8xSU81OP90sJyfuaMqnbdvMnnnGrHHj0Lvp/PPNsrLijqpCKwu9lWrVqlXq99yxY4dt377dDjvsMMvOzjYzs9tvv90uvfTSYrt2WVLsvZXMLAe4mtALaTEw3swWShooaWBRzo12/1XS/5M0Hzg1ShBE+8cDi4ApwCDLW11V9vzvf/CrX4VZTp9/Pkw14Qovt01iyZIwM+3LL4equfvuC6veuUrjzjvv5IEHHgDCwj4333wzHTp04Mgjj+S9994DwiyuN910E+3btyc9PZ3HH38cgA0bNtC1a1fatWvHsccey6uvvgqEKcCPPvporrrqKtq1a8eXX3652z1PPPFEVq1aBUB2djZnn3027du3p3379vwnKslmZ2fTrVs32rVrxxVXXMFhhx3GN998k++1//a3v+2M7Y477gDgxx9/5IwzzqB169a0atWKl156CYBbbrmFli1bkp6ezo033pjndzB37lxOOOEE0tPT6devH999990+fzf7K6mJ9yx0M31jj235jkM3s4sLOjfa/tt93O8vwF+Sia1M2Lo1TK/9/ffw4YdhqU63f2rXhnvugQEDwmC6W28NM8I+9FCYMryy9/4qKXOug+/mFu8167WB44bt92VycnKYNWsWb7zxBnfddRdvvfUWo0eP5qCDDmL27Nls2bKFk08+me7du9O0aVMmTpzIgQceyDfffMMJJ5zAmWeeCcDSpUsZM2YMjz76aJ57TJkyZecAur1N933XXXfRpUsXBg8ezJQpUxg1atTO8xOvPXXqVJYtW8asWbMwM84880zeffddsrOzadSoEa+//joA69at49tvv2XixIksWbIESXz//fd5Yrvwwgt55JFH6Ny5M7fffjt33XUXw6KJEvP73ewvn5W1OPz+9/D++2Gm0mgaYVdMfvaz0EA9bVoYaX7mmdCjR5is8Kij4o7OlaKzzjoLgOOOO46VK1cCMHXqVObPn79ztbh169axbNkymjRpwq233sq7775LlSpVWLVqFV9//TUAhx12GCeccMJu1z711FP5+uuvadCgAffccw+w9+m+33//fSZOnAhAz549qZfwZTDx2lOnTmXq1Km0bdsWCKWZZcuW0bFjR2688UZuvvlmevfuTceOHcnJyaFGjRpceumlnHHGGfTu3Xu3+NatW8f3339P586dgbBa3jnnnLPP383+8uSwv559Ngxwu+EG6N8/7mgqrm7dwgR/I0bAHXeEGW2vvRb+9CeIJllzxaAYvuGXlOrVqwNhvemcaN10M+ORRx6hR48eux379NNPk52dzZw5c0hNTaVZs2Zs3rwZgFq1auW59owZM6hVqxYXX3wxt99+O0OHDt3rdN+2jzE5idc2MwYPHswVV1yR57g5c+bwxhtvMHjwYLp3787tt9/OrFmzmD59OuPGjWP48OG8/fbbSf5m8v/d7C+fW2l/fPxxmI301FNhyJC4o6n4UlPD+hfLloVR10OHhhlgx4zxrq+VVI8ePXjsscd2Tu/96aef8uOPP7Ju3ToaNGhAamoqM2bMSKoH0gEHHMCwYcN49tln+fbbb/c63fcpp5zC+PHjgVA6yK37zy+2p556ig0bNgCwatUq1qxZw+rVq6lZsyYXXHABN954Ix9//DEbNmxg3bp19OrVi2HDhuWZWvyggw6iXr16O9sTxo4du7MUUVK85FBU33wTBm+lpcFLL4WGVFc6GjQIa0oMHBgWIRowAB57DEaODOMkXLmzceNGmjRpsvPzDTfckNR5l156KStXrqRdu3aYGWlpaUyaNInzzz+fX/7yl2RkZNCmTRuOSrIKsmHDhpx33nmMGDGChx9+mEGDBpGenk5OTg6dOnVi5MiR3HHHHZx33nm89NJLdO7cmYYNG1KnTp2dSSBX9+7dWbx4MSeeeCIQuus+99xzLF++nJtuuokqVaqQmprKY489xvr16+nTpw+bN2/GzHjooYfyxPbMM88wcOBANm7cyM9+9jPGjBmT1DMV2d66MZWnV6l3Zd22zaxrV7Pq1c1mzy7de7vd7dgRpglv2DBMET54sNmmTXFHVa6Uha6s5cnmzZtt27ZtZmb2wQcfWOvWreMNKEk+ZXdpuPVWmD49fFvNyIg7mspNCtOEL1oU5m66774wFccHH8QdmaugvvjiC9q3b0/r1q255ppreOKJJ+IOqUR4XUhhTZ4Mf/tbqNL43e/ijsblqls3dHU999ywQt0pp4QG63vuCUuZOldMWrRowSeffBJ3GCXOSw6Fdf/9oRH073+POxKXn+7dYcECuOqq0N01PT1Mne72yXxW3AqtKP++nhwKY9GiMMjt8suhWrW4o3F7U6dO6F78zjtQpQp06RJKej/8EHdkZVKNGjVYu3atJ4gKysxYu3YtNWrUKNR5Xq1UGKNHh+6Uv93r4G5XlnTqFMZG3H57GFn9+uswahScfnrckZUpTZo0ISsrizI1u7ErVjVq1NitN1gyVBG+LWRkZFhmZmbJ3mTrVmjcGDp3DvP9uPLlo49Cl9fchuuhQ+Hgg+OOyrlYSZpjZvn2qvFqpWRNnhzGNlx6adyRuKI4/vgwaPG22+C556BlyzAth3MuX54ckvXkk2HG1W7d4o7EFVX16nD33TB7NjRsGNaNGDAA9hi85Jzz5JCczz+HqVND11Wfirv8a9sWZs0KpYinnw6fZ8+OOyrnyhRPDsl4+unw08c1VBypqaEU8e9/h3UiTjopdFPeXvaXDnGuNHhyKMj27WHZz27doFmzuKNxxS23R9NZZ8HgwXDaaZCVFXdUzsXOk0NB3noLvvgCLrkk7khcSalXD8aNC18CZs8OA+cmTIg7Kudi5cmhIKNHQ/360KdP3JG4kiSFasNPPoEjjghLvl52Gfz4Y9yROReLpJKDpJ6SlkpaLumWfRzXXtJ2Sb8q6FxJf5O0RNJ8SRMl1Y22N5O0SdLc6JXvcqSlIjsbJk0KaxpHi2m4Cq5FC/jPf0IV0+jRYQrwOXPijsq5UldgcpCUAowATgdaAudJarmX44YAbyZ57jSglZmlA58CgxMu95mZtYleA4v0ZMVh7FjYts2rlCqb1FS49154++1QcjjxxDDZoi8o5CqRZEoOHYDlZrbCzLYC44D86lj+D5gArEnmXDObama569nNBAo3trukmYWxDSecAMccE3c0Lg6/+AXMnx/Wrf7DH8KkfqtWxR2Vc6UimeTQGPgy4XNWtG0nSY2BfsCeVUAFnhsZAPwr4XNzSZ9IekdSx/yCknS5pExJmSUyJ8zMmbB4sY+IruwOPhj+8Y/wReHDD6FNm9BJwbkKLpnkoHy27Tkh0zDgZjPbs5N4gedK+iOQAzwfbfoKONTM2gI3AC9IOjDPRcxGmVmGmWWkpaUV/BSF9eSTULt2WB/AVW5SqFr8+GP4yU9CCeLuu72ayVVoyczKmgU0TfjcBFi9xzEZwDhJAIcAvSTlFHSupIuA3kDXaMk6zGwLsCV6P0fSZ8CRQAnPrJdg/fqwLnT//iFBOAfw85+HCfyuvDLM9PrBB2Gepvr1447MuWKXTMlhNtBCUnNJ1YD+wOTEA8ysuZk1M7NmwMvAVWY2aV/nSuoJ3AycaWYbc68lKS1qyEbSz4AWwIr9e8xCeuml0BDpVUpuT7VqwTPPwMiRocG6XbswFYdzFUyBySFqNL6a0AtpMTDezBZKGihpnz2J9nZutHs4UAeYtkeX1U7AfEnzCIlmoJl9W4RnK7onnwyN0McfX6q3deWEBFdcEbq8SmFJ0hEjQicG5yoIX89hTwsWwLHHhvn+r7++eK7pKq5vvw3jYF5/Hc47Lywm5FWRrpzw9RwKw1d7c4Vx8MFhrY977w3VkR06hF5uzpVznhwSbdkCzz4L/frBIYfEHY0rL6pUCSOqp02DtWuhfXt48cW4o3Juv3hySDRpUqgm8BHRrii6dAndXdu2hd/8Bq6+OnzhcK4c8uSQaPRoOOywMG2zc0XRuHHoxfT734dG6k6dwqy+zpUznhxy/fe/oVpgwIBQTeBcUaWmwgMPhGm/lyyBjAx47724o3KuUPyvYK4xY0K3xIsvjjsSV1GcdVYYNFevHnTtGnoyOVdOeHKAsNrbmDHQowccemjc0biK5KijQoLo2jWMjbjqqjDTr3NlnCcHgKlTw9KQPiLalYS6deGf/wwzuz72WGjTKonJIp0rRp4cIIyITkuDX/4y7khcRZWSAkOGhLmYZs0K7RBz58YdlXN75cnh66/DIKYLL4Rq1eKOxlV0558fGqd37ICTToLx4+OOyLl8eXIYOxZycnxsgys9GRkwe3YYD3HuuXDbbT79tytzKndyyF3t7eST4eij447GVSY//WkYD3HJJfCXv0DfvvDDD3FH5dxOlTs5/Oc/sHSplxpcPKpXhyeegOHD4Y03wpK0y5bFHZVzQGVPDm3bwtNPwznnxB2Jq6wkGDQoDMBcsyZM3Dd1atxROVfJk0OtWnDRRT7FsovfqaeGdoimTeH00+GRR+KOyFVylTs5OFeWNG8elh7t3RuuuSa8tu+5LLtzpcOTg3NlSe3a8MorcMMNofTQp09Y09y5UpZUcpDUU9JSScsl3bKP49pL2i7pVwWdK+lgSdMkLYt+1kvYNzg6fqmkHkV9OOfKpZQUePBBePRRmDIFOnaEL7+MOypXyRSYHCSlACOA04GWwHmSWu7luCGE9aKTOfcWYLqZtQCmR5+J9vcHjgF6Ao9G13GucrnyyrD86IoVYT3zOXPijshVIsmUHDoAy81shZltBcYBffI57v+ACcCaJM/tAzwTvX8G6JuwfZyZbTGz/wLLo+s4V/n06BHaIVJTw9oQkybFHZGrJJJJDo2BxDJtVrRtJ0mNgX7AyEKc+xMz+wog+tkg2ftF97xcUqakzGyfxMxVZK1ahZldW7UK04A/+GAYwOlcCUomOSifbXv+lzkMuNnM9uxakcy5RbkfZjbKzDLMLCMtLa2ASzpXzv30pzBjBpx9Ntx4Y6hy8qm/XQmqmsQxWUDThM9NgNV7HJMBjJMEcAjQS1JOAed+LamhmX0lqSG7qqOSuZ9zlU/NmvDSS/DHP8L994e2iH/8Aw46KO7IXAWUTMlhNtBCUnNJ1QiNxZMTDzCz5mbWzMyaAS8DV5nZpALOnQxcFL2/CHg1YXt/SdUlNQdaALOK+oDOVShVqsB994X1zmfMCPOCrVwZd1SuAiowOZhZDnA1oRfSYmC8mS2UNFDSwKKcG+2+H+gmaRnQLfpMtH88sAiYAgzKp7rKucptwIAwzcaqVaEn00cfxR2Rq2BkFaBhKyMjwzIzM+MOw7nSt2QJnHEGrF4N48aFQXPOJUnSHDPLyG+fj5B2rjw76iiYORPS00NPppF7dhh0rmg8OThX3qWlhbUhTj899GK67Tbv6ur2mycH5yqCWrXCALlLLw2LBw0Y4F1d3X5Jpiurc648qFoVRo2CJk3gzjvhq6/g5Zd9SnpXJF5ycK4ikeCOO8Lyt2+9BZ07w//+F3dUrhzy5OBcRXTJJfDqq6E300knwaefxh2RK2c8OThXUZ1xRhgot359SBAzZ8YdkStHPDk4V5F16AAffgh160KXLvDaa3FH5MoJTw7OVXRHHBGm/T7mGOjbFx5/PO6IXDngycG5yqBBg1DF1KMHDBwIt9/uYyHcPnlycK6yqF07NFIPGAB33w2XXQY5OXFH5cooH+fgXGWSmhq6uTZqBPfcA99+Cy+8ADVqxB2ZK2O85OBcZSOFksOwYTBxIvTqBT/8EHdUrozx5OBcZXXttTB2LLz7Lpx6KqxZU/A5rtLwaqU518F3c+OOwrl4/BR4/mhY+Ak82SzM7upVTOVLvTZw3LBiv6yXHJyr7OrXh9atYetW+OQT+HFj3BG5MsBLDiWQcZ0rl1rMD11dt34Gb7wRVphzlVZSJQdJPSUtlbRc0i357O8jab6kuZIyJZ2SsO9aSQskLZR0XcL2l6Lj50paKWlutL2ZpE0J+3z1EudKQ3o6vP9+GE3dtStMmxZ3RC5GBZYcJKUAIwjrPGcBsyVNNrNFCYdNByabmUlKJ6wBfZSkVsBlQAdgKzBF0utmtszMzk24x4PAuoTrfWZmbfbz2ZxzhXX44SFB9OwZ5mZ6/nk455y4o3IxSKbk0AFYbmYrzGwrMA7YbaFaM9tguxajrgXkvj8amGlmG80sB3gH6Jd4riQBvwZeLPpjOOeKTcOG8M47oVrp3HN96dFKKpnk0Bj4MuFzVrRtN5L6SVoCvA4MiDYvADpJqi+pJtALaLrHqR2Br81sWcK25pI+kfSOpI5JPotzrrjUrQtvvhnGQFx5ZVhdzqfbqFSSSQ7KZ1ue/0rMbKKZHQX0Be6Oti0GhgDTgCnAPGDP8frnsXup4SvgUDNrC9wAvCDpwDxBSZdH7RuZ2dnZSTyGc65QatYMg+QuuCCsS3399bBjR9xRuVKSTHLIYvdv+02A1Xs72MzeBQ6XdEj0ebSZtTOzTsC3wM4SgqSqwFnASwnnbzGztdH7OcBnwJH53GeUmWWYWUZaWloSj+GcK7TUVHjmGbjuOvj738MiQtu3xx2VKwXJdGWdDbSQ1BxYBfQHfpN4gKQjCI3IJqkdUA1YG+1rYGZrJB1KSAQnJpx6GrDEzLISrpUGfGtm2yX9DGgBrCjyEzrn9k+VKjB0aKhquvNO+PFHeO45qFYt7shcCSowOZhZjqSrgTeBFOApM1soaWC0fyRwNnChpG3AJuDchAbqCZLqA9uAQWb2XcLl+5O3IboT8GdJOcB2YKCZfVv0R3TO7bfctalr14YbbwwJ4uWX4YAD4o7MlRBZBWhkysjIsMzMzLjDcK5yGDUqrAnxi1+EKcDr1Ik7IldEkuaYWUZ++3z6DOdc4Vx++a4J+7p1g+++K/gcV+54cnDOFd7554dqpU8+8RldKyhPDs65ounbF157DT79FDp1gqysAk9x5YcnB+dc0XXvHgbLrV4NHTvCZ5/FHZErJp4cnHP7p2NHePvtsJpcx46waFHB57gyz5ODc27/ZWSE+ZjMoHNn+PjjuCNy+8mTg3OueLRqBe+9F6bd6NIFPvgg7ojcfvDk4JwrPkccERJEgwahm+tbb8UdkSsiTw7OueJ16KFhDMThh0Pv3mFVOVfueHJwzhW/n/4UZsyAY44JXV5ffTXuiFwheXJwzpWM+vVh+nRo2xZ+9aswaM6VG54cnHMlp27dsBb18cdD//7wwgtxR+SS5MnBOVeyDjwQpkwJYyAuuCCsD+HKPE8OzrmSV7s2vP46nHYa/O538MQTcUfkCuDJwTlXOmrWhMmT4fTTw8yuI0bEHZHbB08OzrnSU6MGvPIK9OkDV18NDz0Ud0RuLzw5OOdKV/Xq8I9/hB5MN9wA998fd0QuH8msIe2cc8UrNRVefDGsQz14MGzdCn/6U1iO1JUJSZUcJPWUtFTSckm35LO/j6T5kuZKypR0SsK+ayUtkLRQ0nUJ2++UtCo6Z66kXgn7Bkf3Wiqpx34+o3OuLKpaFZ59Fi6+OKxPfdttYeI+VyYUWHKQlAKMALoBWcBsSZPNLHFe3unAZDMzSenAeOAoSa2Ay4AOwFZgiqTXzWxZdN5DZvbAHvdrCfQHjgEaAW9JOtLMtu/Xkzrnyp6UFBg9OpQk7r0XtmyBv/3NSxBlQDIlhw7AcjNbYWZbgXFAn8QDzGyD2c6UXwvIfX80MNPMNppZDvAO0K+A+/UBxpnZFjP7L7A8isE5VxFVqQIjR8KgQfDgg3DddV6CKAOSSQ6NgS8TPmdF23YjqZ+kJcDrwIBo8wKgk6T6kmoCvYCmCaddHVVHPSWpXiHvd3lUhZWZnZ2dxGM458qsKlXgkUfg+uvh4Yc9QZQBySSH/Mp3ef7VzGyimR0F9AXujrYtBoYA04ApwDwgJzrlMeBwoA3wFfBgIe83yswyzCwjLS0ticdwzpVpUig53HBDSBDXXusJIkbJ9FbKYvdv+02A1Xs72MzelXS4pEPM7BszGw2MBpB0b3Q9zOzr3HMkPQH8syj3c85VIBI88MCuRGEWEoW3QZS6ZJLDbKCFpObAKkJj8W8SD5B0BPBZ1CDdDqgGrI32NTCzNZIOBc4CToy2NzSzr6JL9CNUQQFMBl6QNJTQIN0CmLUfz+icK0+k0CgNuxLEI494gihlBSYHM8uRdDXwJpACPGVmCyUNjPaPBM4GLpS0DdgEnJvQQD1BUn1gGzDIzL6Ltv9VUhtCldFK4IroegsljQcWEaqgBnlPJecqmdwEkVuSMIPhwz1BlCJZBajTy8jIsMzMzLjDcM4VNzO4+eaQKK66yhNEMZM0x8wy8tvnI6Sdc2WXBEOGhJ9//euuEkQVn/mnpHlycM6VbVKYfyk3UZiFGV09QZQoTw7OubJPgvvuC+9zE8Sjj3qCKEGeHJxz5UNugsgtSZjBY495gighnhycc+WHFOZgyk0UZmHqDU8Qxc6Tg3OufJHgL3/ZlSjM4PHHPUEUM08OzrnyR4J77tmVKKpU8SqmYubJwTlXPklw992wfXtog6ha1cdBFCNPDs658iu3aiknJ4ykrloVhg3zBFEMPDk458q33AFyOTkhMVStumvyPldknhycc+WfBEOHhgQxdGhIELkD51yReHJwzlUMUpjee/v2UJKoWnVXo7UrNE8OzrmKQwqN0jk5oS0iNRXuvDPuqMolTw7OuYold03qnBy4665QgrjttrijKnc8OTjnKp4qVeCJJ0IV05/+FBLELbfEHVW54snBOVcxpaTAU0+FEsTgwSFB3Hhj3FGVG54cnHMVV0oKPPNMSBA33RQSxHXXxR1VuZDUWHNJPSUtlbRcUp6ymaQ+kuZLmispU9IpCfuulbRA0kJJ1yVs/5ukJdF5EyXVjbY3k7QputZcSSP3/zGdc5VW1arw3HNw9tlw/fVhLQhXoAKTg6QUYARwOtASOE9Syz0Omw60NrM2wADgyejcVsBlQAegNdBbUovonGlAKzNLBz4FBidc7zMzaxO9Bhb14ZxzDgi9ll58Efr2hauvDg3Wbp+SKTl0AJab2Qoz2wqMA/okHmBmG2zXYtS1gNz3RwMzzWyjmeUA7wD9onOmRtsAZgJN9u9RnHNuH1JT4aWX4Je/hCuvDO0Rbq+SSQ6NgS8TPmdF23YjqZ+kJcDrhNIDwAKgk6T6kmoCvYCm+dxjAPCvhM/NJX0i6R1JHfMLStLlURVWZnZ2dhKP4Zyr9KpVg3/8A3r0gEsvhRdeiDuiMiuZ5JDf8ELLs8FsopkdBfQF7o62LQaGEKqQpgDzgJzE8yT9Mdr2fLTpK+BQM2sL3AC8IOnAfO43yswyzCwjLS0ticdwzjmgenV45RXo3BkuvBAmTIg7ojIpmeSQxe7f9psAq/d2sJm9Cxwu6ZDo82gza2dmnYBvgWW5x0q6COgNnJ9bLWVmW8xsbfR+DvAZcGShnso55/alZk147TU4/njo3x/++c+4IypzkkkOs4EWkppLqgb0ByYnHiDpCClMYCKpHVANWBt9bhD9PBQ4C3gx+twTuBk408w2JlwrLWoER9LPgBbAiv15SOecy6N2bXjjDWjTJvRkmjo17ojKlALHOZhZjqSrgTeBFOApM1soaWC0fyRwNnChpG3AJuDchAbqCZLqA9uAQWb2XbR9OFAdmBbllZlRz6ROwJ8l5QDbgYFm9m0xPa9zzu1y0EHw5pvQpUvoyfSvf4XqJod2/Q0vvzIyMiwzMzPuMJxz5VV2NvziF/D556EEcdJJcUdUKiTNMbOM/Pb5gqvOOZeWBm+9BY0awemng3/Z9OTgnHMANGwI06fDwQdD9+4wb17cEcXKk4NzzuVq2hTefhtq1YJu3WDRorgjio0nB+ecS9S8eShBpKTAaafBsmUFn1MBeXJwzrk9HXlkSBDbtkHXrrByZdwRlTpPDs45l5+WLUMj9YYNoatrVlbcEZUqTw7OObc3rVuHcRBr14YSxNdfxx1RqfHk4Jxz+9K+Pbz+eig5dOsG31aOMbmeHJxzriCnnAKvvgpLl4ZxED/8EHdEJc6Tg3POJeO00+Dll+Hjj8OaEBs3FnxOOebJwTnnkvXLX4YlR997D846C7ZsiTuiEuPJwTnnCuPcc+HJJ0NDdf/+obtrBeTJwTnnCmvAAPj732HSJLj4Yti+Pe6Iil2BU3Y755zLxzXXwI8/wq23huk2Hn8clN/CmeWTJwfnnCuqwYPDILl77w2LBz34YIVJEJ4cnHNuf9xzT0gQDz0EderAXXfFHVGx8OTgnHP7QwqJ4ccf4c9/DlVMf/hD3FHtt6QapCX1lLRU0nJJt+Szv4+k+ZLmSsqUdErCvmslLZC0UNJ1CdsPljRN0rLoZ72EfYOjey2V1GM/n9E550pWlSqhzaF/f7j5Znj00bgj2m8FJgdJKcAI4HSgJXCepJZ7HDYdaG1mbYABwJPRua2Ay4AOQGugt6QW0Tm3ANPNrEV0/i3ROS2B/sAxQE/g0SgG55wru1JS4Nlnw1iIQYPgmWfijmi/JFNy6AAsN7MVZrYVGAf0STzAzDbYrsWoawG5748GZprZRjPLAd4B+kX7+gC5v71ngL4J28eZ2RYz+y+wPIrBOefKttRUGD8+jKYeMCCMqC6nkkkOjYEvEz5nRdt2I6mfpCXA64TSA8ACoJOk+pJqAr2AptG+n5jZVwDRzwaFuZ9zzpVJNWqE8Q8nngi/+Q1MmRJ3REWSTHLIr1+W5dlgNtHMjiKUAO6Oti0GhgDTgCnAPCCnOO4n6fKofSMzOzu7gEs651wpqlUL/vlPOOaYMM3G++/HHVGhJZMcstj1bR+gCbB6bweb2bvA4ZIOiT6PNrN2ZtYJ+BbIXXPva0kNAaKfawpzPzMbZWYZZpaRlpaWxGM451wpqls3TLFx6KFwxhlhwr5yJJnkMBtoIam5pGqExuLJiQdIOkIKIz8ktQOqAWujzw2in4cCZwEvRqdNBi6K3l8EvJqwvb+k6pKaAy2AWUV7POeci1GDBjBtWkgUPXrAkiVxR5S0Asc5mFmOpKuBN4EU4CkzWyhpYLR/JHA2cKGkbcAm4NyEBuoJkuoD24BBZvZdtP1+YLykS4AvgHOi6y2UNB5YRKiCGmRmFW/iEudc5dC0aVhutGPH0FD9/vvQrFncURVIu/6Gl18ZGRmWmZkZdxjOObd38+dD585Qv36Y8rthw7gjQtIcM8vIb5/Pyuqcc6UhPR3+9S/43/+ge/cyv9yoJwfnnCstJ5wQlhv99NOw3Oj69XFHtFeeHJxzrjR17RoGys2ZA336wObNcUeUL08OzjlX2vr0gaefhhkz4Ne/LpOryXlycM65OFxwAYwYAa+9FlaT27Ej7oh241N2O+dcXK66CtatC6vJHXhgmM21jCwW5MnBOefiNHhwSBBDhsBBB8H998cdEeDJwTnn4nffffDDDyFB1KsX1oSImScH55yLmwTDh8N338Ett8DBB8Nll8UakicH55wrC6pUCQsErVsHV1wRqph+/ev4wontzs4553ZXrVpYIOikk0JvpjffjC0UTw7OOVeW1KwZ1oJo2TKsBfHhh7GE4cnBOefKmty1IBo1gl69wqR9pcyTg3POlUU/+UlYC6JWrbAWxGeflertPTk451xZ1awZTJ0aptfo1g1W73URzmLnycE558qyli3DVN/Z2aU61bcnB+ecK+vat4fJk2H58tAGsWFDid/Sk4NzzpUHp54K48bB7NmhF9OWLSV6u6SSg6SekpZKWi7plnz295E0X9JcSZmSTknYd72khZIWSHpRUo1o+0vR8XMlrZQ0N9reTNKmhH0ji+lZnXOufOvbF0aPDg3V558P27eX2K0KHCEtKQUYAXQDsoDZkiab2aKEw6YDk83MJKUD44GjJDUGrgFamtkmSeOB/sDTZnZuwj0eBNYlXO8zM2uzn8/mnHMVz8UXw/ffw/XXh5HUTzxRIjO5JjN9RgdguZmtAJA0DugD7EwOZpZYAVYLsD3ucYCkbUBNYLfmdkkCfg10KcoDOOdcpXPddaFh+u67wzxMf/1rsd8imeTQGPgy4XMWcPyeB0nqB9wHNADOADCzVZIeAL4ANgFTzWzqHqd2BL42s2UJ25pL+gT4AbjNzN7L536XA5cDHHrooUk8hnPOVSB33RXmYWrRokQun0ybQ37lFcuzwWyimR0F9AXuBpBUj1DKaA40AmpJumCPU88DXkz4/BVwqJm1BW4AXpB0YD73G2VmGWaWkZaWlsRjOOdcBSLB3/9eYrO3JpMcsoCmCZ+bsEfVUCIzexc4XNIhwGnAf80s28y2Aa8AJ+UeK6kqcBbwUsL5W8xsbfR+DvAZcGTST+Scc26/JZMcZgMtJDWXVI3QoDw58QBJR0RtB0hqB1QD1hKqk06QVDPa3xVYnHDqacASM8tKuFZa1AiOpJ8BLYAVRX1A55xzhVdgm4OZ5Ui6GngTSAGeMrOFkgZG+0cCZwMXRo3Om4BzzcyAjyS9DHwM5ACfAKMSLt+f3auUADoBf5aUA2wHBppZ6QwJdM45B4DC3/DyLSMjwzIzM+MOwznnyhVJc8wsI799PkLaOedcHp4cnHPO5eHJwTnnXB6eHJxzzuVRIRqkJWUDnxdw2CHAN6UQTllVmZ/fn73yqszPn8yzH2Zm+Y4irhDJIRmSMvfWKl8ZVObn92evnM8Olfv59/fZvVrJOedcHp4cnHPO5VGZksOogg+p0Crz8/uzV16V+fn369krTZuDc8655FWmkoNzzrkkeXJwzjmXR6VIDpJ6SloqabmkW+KOpzRJekrSGkkL4o6ltElqKmmGpMWSFkq6Nu6YSoukGpJmSZoXPftdccdU2iSlSPpE0j/jjqW0SVop6f9JmiupSLOSVvg2h2htiE+BboSFi2YD55nZon2eWEFI6gRsAJ41s1Zxx1OaJDUEGprZx5LqAHOAvpXh3z5aP6WWmW2QlAq8D1xrZjNjDq3USLoByAAONLPeccdTmiStBDLMrMgDACtDyaEDsNzMVpjZVmAcYenSSiFama9SrodhZl+Z2cfR+/WEhaYaxxtV6bBgQ/QxNXpV7G+CCSQ1Iaxl/2TcsZRXlSE5NAa+TPicRSX5A+F2kdQMaAt8FHMopSaqVpkLrAGmmVmleXZgGPAHYEfMccTFgKmS5ki6vCgXqAzJQflsqzTfoBxIqg1MAK4zsx/ijqe0mNl2M2tDWPe9g6RKUa0oqTewJlqDvrI62czaAacDg6Lq5UKpDMkhC2ia8LkJsDqmWFwpi+rbJwDPm9krcccTBzP7Hvg30DPeSErNycCZUb37OKCLpOfiDal0mdnq6OcaYCKher1QKkNymA20kNRcUjXCutWTY47JlYKoUXY0sNjMhsYdT2mSlCapbvT+AOA0YEmsQZUSMxtsZk3MrBnh//e3zeyCmMMqNZJqRR0wkFQL6A4UurdihU8OZpYDXA28SWiQHG9mC+ONqvRIehH4EPi5pCxJl8QdUyk6Gfgt4Zvj3OjVK+6gSklDYIak+YQvSNPMrNJ16aykfgK8L2keMAt43cymFPYiFb4rq3POucKr8CUH55xzhefJwTnnXB6eHJxzzuXhycE551wenhycc87l4cnBOedcHp4cnHPO5fH/AbH6/HhCxoVaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#细化学习曲线\n",
    "alpharange = np.arange(0.1,5,0.2)\n",
    "ridge_re = []\n",
    "linear_re = []\n",
    "for alpha in alpharange:\n",
    "    ridge = Ridge(alpha=alpha)\n",
    "    linear = LinearRegression()\n",
    "    ridge_r2 = cross_val_score(ridge,X,y,cv=5,scoring = \"r2\").mean()\n",
    "    linear_r2 = cross_val_score(linear,X,y,cv=5,scoring = \"r2\").mean()\n",
    "    ridge_re.append(ridge_r2)\n",
    "    linear_re.append(linear_r2)\n",
    "plt.plot(alpharange,ridge_re,color=\"red\",label=\"Ridge\")\n",
    "plt.plot(alpharange,linear_re,color=\"orange\",label=\"LinearRegression\")\n",
    "plt.title(\"Mean\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出，在鲍鱼数据集上，随着$\\alpha$的增大，岭回归的结果轻微上升，随后骤降。说明，鲍鱼数据集带有很轻微的一部分共线性，这种共线性被正则化参数$\\alpha$消除后，模型的效果提升了一点点，但是对于整个模型而言是杯水车薪。在过了控制多重共线性的点后，模型的效果飞速下降，显然是正则化的程度太重，挤占了参数$w$本来的估计空间。从这个结果可以看出，鲍鱼数据集的核心问题不在于多重共线性，岭回归不能够提升模型表现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.2 LASSO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了岭回归之外，最常被人们提到还有模型Lasso。Lasso全称最小绝对收缩和选择算子（least absolute shrinkage and selection operator），由于这个名字过于复杂所以简称为Lasso。和岭回归一样，Lasso是被创造来作用于多重共线性问题的算法，不过Lasso使用的是系数$w$的L1范式（L1范式则是系数$w$的绝对值）乘以正则化系数$\\alpha$，所以Lasso的损失函数表达式为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\min\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2 + \\alpha {||\\boldsymbol{w}||_1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "许多博客和机器学习教材会说，Lasso与岭回归非常相似，都是利用正则项来对原本的损失函数形成一个惩罚，以此来防止多重共线性。这种说法不是非常严谨，**Lasso不是从根本上解决多重共线性问题，而是限制多重共线性带来的影响**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "比较有意思的是，尽管LASSO是为了限制多重共线性被创造出来的，然而世人其实并不使用它来抑制多重共线性，反而接受了它在其他方面的优势。事实上，岭回归和Lasso的不同就是L1和L2正则化，而这两个正则化的核心差异就是他们对系数$w$的影响：两个正则化都会压缩系数$w$的大小，对标签贡献更少的特征的系数会更小，也会更容易被压缩。不过，L2正则化只会将系数压缩到尽量接近0，但L1正则化主导稀疏性，因此会将系数压缩到0。这个性质，让Lasso成为了线性模型中的特征选择工具首选，接下来，我们就来看看如何使用Lasso来选择特征。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn中我们使用类Lasso来调用lasso回归，众多参数中我们需要比较在意的就是参数$\\alpha$，正则化系数。另外需要注意的就是参数positive。当这个参数为\"True\"的时候，是我们要求Lasso回归出的系数必须为正数，以此来保证我们的$\\alpha$一定以增大来控制正则化的程度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*class* `sklearn.linear_model.Lasso`(*alpha=1.0*, *fit_intercept=True*, *normalize='deprecated'*, *precompute=False*, *copy_X=True*, *max_iter=1000*, *tol=0.0001*, *warm_start=False*, *positive=False*, *random_state=None*, *selection=’cyclic’*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:00:56.421180Z",
     "start_time": "2024-03-06T03:00:56.413342Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import Ridge, LinearRegression, Lasso\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:01:02.552555Z",
     "start_time": "2024-03-06T03:01:02.533804Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "      <th>年龄</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   性别     长度     直径     高度    整体重量     肉重量    内脏重量     壳重  年龄\n",
       "0   1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1   1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2  -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3   1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4   0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入数据集\n",
    "data = pd.read_table('abalone.txt',header=None)\n",
    "data.columns=['性别','长度','直径','高度','整体重量','肉重量','内脏重量','壳重','年龄']\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:01:04.974932Z",
     "start_time": "2024-03-06T03:01:04.951063Z"
    }
   },
   "outputs": [],
   "source": [
    "# 提取特征和标签\n",
    "X = data.iloc[:,:-1]\n",
    "y = data.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:01:07.099144Z",
     "start_time": "2024-03-06T03:01:07.086214Z"
    }
   },
   "outputs": [],
   "source": [
    "# 切分训练集和测试集\n",
    "Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:01:17.779681Z",
     "start_time": "2024-03-06T03:01:17.767212Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8.336487137622806,\n",
       " -119.13820060178554,\n",
       " 1346.6290126335914,\n",
       " 812.2086299064794,\n",
       " 926.5477237852635,\n",
       " -2045.424714682015,\n",
       " -979.4264604252113,\n",
       " 945.6900963011188]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#线性回归进行拟合\n",
    "reg = LinearRegression().fit(Xtrain,Ytrain)\n",
    "(reg.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8.336487137623426,\n",
       " -119.13820060181226,\n",
       " 1346.6290126336187,\n",
       " 812.2086299064923,\n",
       " 926.5477237853063,\n",
       " -2045.424714682036,\n",
       " -979.4264604252289,\n",
       " 945.690096301087]"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#岭回归进行拟合\n",
    "Ridge_ = Ridge(alpha=0).fit(Xtrain,Ytrain)\n",
    "(Ridge_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:02:59.481736Z",
     "start_time": "2024-03-06T03:02:59.196709Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/4t/jjfxrq9d5sqc5fk9qcn241_c0000gn/T/ipykernel_40819/1028009299.py:2: UserWarning: With alpha=0, this algorithm does not converge well. You are advised to use the LinearRegression estimator\n",
      "  lasso_ = Lasso(alpha=0).fit(Xtrain,Ytrain)\n",
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:647: UserWarning: Coordinate descent with no regularization may lead to unexpected results and is discouraged.\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 7.052e+03, tolerance: 3.019e+00 Linear regression models with null weight for the l1 regularization term are more efficiently fitted using one of the solvers implemented in sklearn.linear_model.Ridge/RidgeCV instead.\n",
      "  model = cd_fast.enet_coordinate_descent(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[8.336488107275883,\n",
       " -119.1381649596776,\n",
       " 1346.6290666881614,\n",
       " 812.2086526405136,\n",
       " 926.5472393227017,\n",
       " -2045.424315599499,\n",
       " -979.4259348196874,\n",
       " 945.6907203292689]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Lasso进行拟合\n",
    "lasso_ = Lasso(alpha=0).fit(Xtrain,Ytrain)\n",
    "(lasso_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso报出的3条警告分别是：\n",
    "- 1. 正则化系数为0，这样算法不可收敛！如果你想让正则化系数为0，请使用线性回归吧\n",
    "- 2. 没有正则项的坐标下降法可能会导致意外的结果，不鼓励这样做！\n",
    "- 3. 目标函数没有收敛，你也许想要增加迭代次数，使用一个非常小的alpha来拟合模型可能会造成精确度问题！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看到这三条内容，大家可能会比较懵——怎么出现了坐标下降？这是由于sklearn中的Lasso类不是使用最小二乘法来进行求解，而是使用坐标下降。使用坐标下降，就有迭代和收敛的问题，因此sklearn不推荐我们使用0这样的正则化系数。如果我们的确希望取到0，那我们可以使用一个比较很小的数，比如0.01，或者$10*e^{-3}$这样的值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:05:27.294672Z",
     "start_time": "2024-03-06T03:05:27.264841Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4.818727347188201,\n",
       " 0.0,\n",
       " 700.3960115931422,\n",
       " 0.0,\n",
       " 468.51821404108176,\n",
       " -1399.812441395366,\n",
       " -0.0,\n",
       " 1350.554084065464]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Lasso进行拟合\n",
    "lasso_ = Lasso(alpha=0.01).fit(Xtrain,Ytrain)\n",
    "(lasso_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:05:33.047562Z",
     "start_time": "2024-03-06T03:05:33.028874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-1.3729984733843392,\n",
       " 5.673063480703414,\n",
       " 4.853042870222797,\n",
       " 1.9720920903161372,\n",
       " 22.48785528399166,\n",
       " 7.645682997198488,\n",
       " 4.648396242621463,\n",
       " 7.629806980161298]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#加大正则项系数，观察模型的系数发生了什么变化\n",
    "Ridge_ = Ridge(alpha=10**4).fit(Xtrain,Ytrain)\n",
    "(Ridge_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso_ = Lasso(alpha=10**4).fit(Xtrain,Ytrain)\n",
    "(lasso_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4.818727347188201,\n",
       " 0.0,\n",
       " 700.3960115931422,\n",
       " 0.0,\n",
       " 468.51821404108176,\n",
       " -1399.812441395366,\n",
       " -0.0,\n",
       " 1350.554084065464]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#看来10**4对于Lasso来说是一个过于大的取值\n",
    "lasso_ = Lasso(alpha=0.01).fit(Xtrain,Ytrain)\n",
    "(lasso_.coef_*100).tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:06:03.661021Z",
     "start_time": "2024-03-06T03:06:03.634864Z"
    }
   },
   "outputs": [],
   "source": [
    "reg = LinearRegression().fit(Xtrain,Ytrain)\n",
    "Ridge_ = Ridge(alpha=0.1).fit(Xtrain,Ytrain)\n",
    "lasso_ = Lasso(alpha=0.1).fit(Xtrain,Ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEJCAYAAACQZoDoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/gklEQVR4nO3dd3hUZdrH8e+dQkIvoRPAUKUIEUJRBBFREGmyKrh2VNRFV8R1xdX1RV3XVVHU1cV1sYsgIqgoIIoCiqAEpIROqAGE0EsSUuZ+/zgTSEwIIZnJmUnuz3WdayanzW9CmHvOec7zHFFVjDHGmGwhbgcwxhgTWKwwGGOMycUKgzHGmFysMBhjjMnFCoMxxphcrDAYY4zJxSeFQUTeFpF9IpKQY95YEdklIiu8U78cyx4Vkc0iskFE+uSY31FEVnuXvSoi4ot8xhhjCs9XRwzvAn3zmT9eVWO90ywAEWkNDAPaeLf5j4iEetefAIwAmnun/PZpjDHGj8J8sRNVXSgi5xVy9UHAFFU9CWwVkc1AZxHZBlRR1cUAIvI+MBiYXdDOatasqeedV9iXNsYYA7Bs2bL9qlorv2U+KQwFuE9EbgHigYdU9RDQAFiSY50k77wM7/Pfz89DREbgHFnQqFEj4uPj/RDdGGNKLxHZfqZl/mx8ngA0BWKBPcCL2XnyWVcLmJ93puqbqhqnqnG1auVb8IwxxhSR3wqDqu5V1SxV9QD/Azp7FyUBDXOsGg3s9s6Pzme+McaYEuS3wiAi9XL8eA2QfcXSF8AwEYkQkRicRuZfVHUPcExEunqvRroF+Nxf+YwxxuTPJ20MIjIZ6AnUFJEk4P+AniISi3M6aBtwN4CqrhGRqcBaIBMYqapZ3l3di3OFU3mcRucCG57PJCMjg6SkJNLS0or4joJbZGQk0dHRhIeHux3FGBOEJNiH3Y6Li9PfNz5v3bqVypUrExUVRVnrCqGqHDhwgGPHjhETE+N2HGNMgBKRZaoal9+yUtnzOS0trUwWBQARISoqqsweLRljiq9UFgagTBaFbGX5vRtjiq/UFgbjop074Z13IDPT7STGmCKwwuAnlSpVyjNv7NixNGjQgNjYWFq3bs3kyZNdSFYC7rwThg+HHj1g61a30xhjzpEVhhL24IMPsmLFCj7//HPuvvtuMjIy3I7kWz/+CHPnwpAhsGYNtG8PH37odipjzDmwwuCS5s2bU6FCBQ4dOuR2FN/6v/+D2rXh/fdh5Upo1w5uvhluugmOHHE7nTGmEPw9VpL7Ro2CFSt8u8/YWHj55WLtYvny5TRv3pzatWv7JFJAmD8fvvsOxo+HihWdaf58+Oc/4amnYNEi+OgjuOgit5MaYwpgRwwlbPz48bRs2ZIuXbowduxYt+P4jio88QTUqwd33316fliYM3/hQufn7t2dImEN08YErNJ/xFDMb/a+9uCDD/KXv/yF6dOnc8stt5CYmEhkZKTbsYpv3jz44Qf497+hfPm8yy++2DlyGznSOd00dy5MmgSNG5d4VGNMweyIwSVDhgwhLi6O9957z+0oxZd9tBAdDXfddeb1qlZ1GqI/+ABWrXIapqdMKbmcxphCscLgJykpKURHR5+aXnrppTzrPPHEE7z00kt4PB4XEvrQnDmweDE8/jhERJx9/Ztuco4eWrWCG26AW2+FY8f8HtMYUzil/1SSSwrzYd+xY0c2bNhQAmn8KPto4bzz4PbbnXn7l8C2SXD+g1CpSf7bNWninHp66il45pnTDdOdO+e/vjGmxNgRgymeL7+E+Hj4+9+hXDnITIFFf4SNr8GX58Oy0XDyYP7bhoU5hWH+fMjIcNohnnkGsrLyX98YUyKsMJii83ico4WmTZ2+CgAJ/4ATW+HiyRBzC2x4Gb5oCutehKyT+e+ne3enz8O11zqno3r1cobVMMa4wgqDKbrPPnPaCp54AsLD4fAaWPcCxNwK5w2DLhOh30qo2RV+/Qt82Qq2TXFOP/1etWoweTK8+y4sW+Z0jPvkk5J9P8YYwAqDKSqPx7nstGVL+OMfQT2w9B4IrwIXjju9XrUL4LLZcNlcZ9lPN8DXXWDfwrz7FHEaolesgObN4frr4Y474PjxEntbxhgrDKaoPvkEEhKc4hAWBlvegeQf4cIXILJm3vXrXQF9l0HXdyF1N3x7KSwcDEfzaXxv1sxpjP7b35xRWjt0cNoxjDElwgqDOXdZWTB2LLRu7XyrT0uGX/8KtS6BJredebuQUGhyKwzYCO2fgd++g6/awNKRkLYv97rh4U5D9HffQWqqM4zGc885RyrGGL+ywuAnoaGhxMbG0rZtWwYMGMDhw4cB2L17N9dee22+2/Ts2ZPf36Y0IE2ZAuvXw5NPQmgo/PowZByFTm+AFOJPKqwCtPkbDNwMze6Bzf+FL5pBwjPOVU059ezpNEwPHgxjxsAVV8CuXf54V8YYLysMflK+fHlWrFhBQkICNWrU4PXXXwegfv36TJs2zeV0xZCZ6RSEdu2cobX3zoet70Grh6Fam3PbV2Rt6PQaXL0G6l4Oqx6HmS1gy7vgyXHJao0aMHUqTJwIS5Y4rz1jhi/flTEmB58UBhF5W0T2iUhCjnk1ROQbEdnkfayeY9mjIrJZRDaISJ8c8zuKyGrvslellNyj8qKLLmKX91vutm3baNu2LQCpqakMGzaMdu3aMXToUFJTU09t89Zbb9GiRQt69uzJXXfdxX333QdAcnIyf/jDH+jUqROdOnVi0aJFJftmPvwQNm1yioNmOA3OFWOg7eNF32eVltBjBvReCBUawJLbYU4H2DP39DoiTkP0r79CTIxTlO6+G06cKP57Msbk4quez+8CrwHv55g3Bpinqv8SkTHenx8RkdbAMKANUB/4VkRaqGoWMAEYASwBZgF9gdnFSrZsFBxaUaxd5FE9Fjq+XKhVs7KymDdvHnfccUeeZRMmTKBChQqsWrWKVatW0aFDB8A53fT000+zfPlyKleuTK9evWjfvj0ADzzwAA8++CCXXHIJO3bsoE+fPqxbt85X76xgGRlOh7QOHWDQIKfPwtEN0HO2c3qouGp3hyuXwI6psOJR+L4P1OsDsc9D9XbOOi1awE8/OR3qXngBFixwLnO98MLiv74xBvDREYOqLgR+3711EJA9Qtx7wOAc86eo6klV3QpsBjqLSD2giqouVlXFKTKDCVKpqanExsYSFRXFwYMHueKKK/Kss3DhQm666SYA2rVrR7t2zoffL7/8wqWXXkqNGjUIDw/nuuuuO7XNt99+y3333UdsbCwDBw7k6NGjHCupcYbefde5VedTT8GxzbDmGWh0PdTv67vXEIHGQ6H/OrjwRTjwC8yOhSXDIcXbtlCunNMQ/e23zhhLXbrAiy9aw7QxPuLPsZLqqOoeAFXdIyLZd6RpgHNEkC3JOy/D+/z38/MQkRE4RxY0atSo4BSF/Gbva9ltDEeOHKF///68/vrr/PnPf86zXn5nyzS/DmBeHo+HxYsXUz6/oa396eRJ+Mc/nA/hq66C+X0gNMJ/v9/QCGg1Gpre7jRKb/w3bJ8C5z8Erf8K4ZWdHtKrVjn3mP7LX+Drr+G995x7QhhTmqkHlj3oXOVXo4PPd+9G43N+7QZawPy8M1XfVNU4VY2rVauWT8P5WtWqVXn11VcZN25cnvs79+jRg0mTJgGQkJDAqlWrAOjcuTMLFizg0KFDZGZm8umnn57a5sorr+S111479fMKX9+d7kzefht27HCOFnZMgd++hfb/hPJ+/hAuVx06jIP+6yF6EKz5B8xsBpveAE8mREXB9Onw3/8695u+4AL44gv/ZjLGbXu/h42vwhH/nEb2Z2HY6z09hPcx+0L1JKBhjvWigd3e+dH5zA96F154Ie3bt2fK7+49cO+993L8+HHatWvH888/T2fvyKINGjTgb3/7G126dKF37960bt2aqlWrAvDqq68SHx9Pu3btaN26NW+88Yb/30BamtOnoFs3uDQOlj8INeKcS01LSqUY6DYZrvwZKreEpffCrAsgyVsERoyA5cuhYUOn/eNPf4KUlIL3aUywSpwIIVUg+hr/7F9VfTIB5wEJOX5+ARjjfT4GeN77vA2wEogAYoAtQKh32VKgK87Rw2yg39let2PHjvp7a9euzTMv2Bw7dkxVVTMyMrR///46ffr0c9rep7+DV15RBdV581R/vkf1oxDVA8t8t/9z5fGo7vxMdWZL1UmofnOp6v6lzrK0NNWHHnLytm6tunKlezn9weNxJlN2pe1X/aic6i2oPv98kXcDxOsZPld9dbnqZGAx0FJEkkTkDuBfwBUisgm4wvszqroGmAqsBeYAI9W5IgngXmAiToN0IsW9IimIjR079lQHuZiYGAYPHuxOkJQUePZZp6NZuwpOZ7QWf/bLec1CE3FOK/VbDXGvw5G18HUnZ7jvjD0wbpzT3nDwIHTq5NzeNdgaptPTYd06p7/Gv/7l3Ovi4ouhZk2nx3mwvR/jO1s/AE2H+UCOC1N8SbSAhs5gEBcXp7/vLbxu3TpatWrlUqLA4LPfwUsvwUMPwfx5cOJBSD8IV691Gn8DRcZRWPs8rH/RaZRr+WenZ/WRTKfvw8yZ0Levc1VVnTpupz1NFfbtgw0bnGn9+tPPt27NfV+KevWcAQvDwpyrsVaudDr6mbJF1TmFmrAJvu0B33xT5F2JyDJVjctvmd3BzZzZ8ePOt9XevaH2cvh1FXSfHlhFAZxRW9v/A5rfA6v+7tz7IfFtaPt3mP4JvPmWU9wuuMAZlO/qq0s2X1oabN58+kM/ZxE4cuT0epGRzqiyF14Iw4ZByxiIqQS1FbJ2O5cIJ6+GZsD8760wlEUHfoYja+Br4O47/fYyVhjMmb3+OiQnw9/vhVU3Q4MBED3Y7VRnViEaur4DLUc54zctf9C5zPXqZ6HHUrjxRujfH+6/H55/3vkg9hVV+O23vN/8N2yAbdtyn/pp0OD0cOUtY6BZFagrUP4IHN/sFIBj70PKTtiusN27XXhVKFcNugOzvgIe8F1+Exw2/w8yw2B9ZWf8MD+xwmDyd/So8+F5VV/Qd515cf92zu8Huurtoddc2P01rHgYFg2FqK4wazy88AW88gp8/73TY9o7PEmhpaY6Q4Lkd/onZ0fD8uWdD/9OneDm66FFFagfClVSIWOn98P/C0jdBUdwJoCIKKjUzOkFXqkZVG4GlZs7j+VqwNH18FVrOLTIKTYhNtxZmZFx1OnL85MHrrsZIiL89lJWGEz+Xn3Vabx9uBfs+qtzn4WKjd1OdW7q94G6vZ1B/lb9HRZcDkOugSv+B8Mfg7g4p6F65MjcBU/VGcE157f+7CKwY0fuO9A1bOgUgNuHwvk1IDoMaqRDSDIcT4RjCyDtY2fd3d4psrbzoV+3t/OBf6oANHX6bRSkyvmg1SDmMKxeDd6hUkwZsP1jyEqBecCneYfY8SUrDH5SqVIljgfrnccOH3aGmLjmKjjyqnMXtpZBetoiJBSaDneG2Vg/HtY+B1kz4ZNbYPw257TS7NnQufPpArBxY+67xlWs6Hz4XxoHra6ExhFQMxMijkDaNji2Ek5+66x7zDuVr+982Nfv5/3Qzy4ATZ02kaISgdo9ofVn8P13VhjKks3/g+QIqNbW7+1LVhhMXi+/7BSHETXg4C7oNhVCwt1OVTxhFZ0RYJveBavHQuL/4I8VoV9/GD3XKQ6NGkK7JtDnKogp7zT6VjgGHm/Db/pyZ18KJAMVGjof+NGD8374h1X033tpMgCSP4M5M4EH/fc6JnAcWgkHlzoX8N95l99fzgpDCZo5cyb/+Mc/SE9PJyoqikmTJlGnTh0WLFjAAw8438hFhIULF3L8+HGGDh3K0aNHyczMZMKECXTv3p3Jkyfzz3/+E1Xl6quv5rnnnvNtyIMHYfx4uL0XHJoMzUZArYt8+xpuKl8HOk9wLmld8QhkzIT36kNkHTixBTLmn143JQRo5HzgN7r+9Id/5ebOUONhJTxeVbY6vZzHo0usnaGsSHwLPCGwrBx8PMzvL1fqC8OoUaN8Pp5QbGwsL7/88jlvd8kll7BkyRJEhIkTJ/L888/z4osvMm7cOF5//XW6devG8ePHiYyM5M0336RPnz489thjZGVlkZKSwu7du3nkkUdYtmwZ1atX58orr+Szzz7zbee3F1+E40dhwD7QmhD7rO/2HUiqtoJLv4C9C2DNP51TNLUuPt3QW7kZVDzPGcwv0FQ6D7QWxCQ7gwjGxrqdyPhTZqrTqW1ZCPS7HrzD4/hTqS8MgSQpKYmhQ4eyZ88e0tPTiYmJAaBbt26MHj2aG2+8kSFDhhAdHU2nTp0YPnw4GRkZDB48mNjYWL777jt69uxJ9sCBN954IwsXLvRdYdi/37li5+EOkLocLp509sbQYFfnUmcKNvUuh1ZT4Pt5VhhKu53TIeMwzAUm+LfROVupLwxF+WbvL/fffz+jR49m4MCBzJ8/n7FjxwIwZswYrr76ambNmkXXrl359ttv6dGjBwsXLuSrr77i5ptv5uGHH6ZKlWI0WhbG889DuRMQuxFq94bGN/j39UzRNRkAv02Bn74AHnI7jfGnxIlwJBIyo6F79xJ5STs5WYKOHDlCgwbOLSbee++9U/MTExO54IILeOSRR4iLi2P9+vVs376d2rVrc9ddd3HHHXewfPlyunTpwoIFC9i/fz9ZWVlMnjyZSy/10bfdvXvhtdfgb42ADOg0ITj6LJRVdS5zHo//knvoDFO6HN0E++bDnDQYfmeJ/Z8s9UcMbklJSSE6+vQo4qNHj2bs2LFcd911NGjQgK5du7J161bAOar5/vvvCQ0NpXXr1lx11VVMmTKFF154gfDwcCpVqsT7779PvXr1ePbZZ7nssstQVfr168egQYN8E/i556DlSai/A9o85ZxjN4GrfD3Q+tBkt9POYLc2LZ22vAUqsEjgv7eW2MvaIHql1Dn9DnbvhvObwEvloG59uGplYDa6mtzm3wZb3oPDz8Poh91OY3zNkwEzGsKyw7CmD3z+uU93X9AgenYqyTjDavdLd67Z7zTBikKwaDoQIoEE335gmACx60s4uRfmnHRGCS5BVhjKup074cv/Qn+BmFtOn7s2ga/2pU5nu9R4a2cojRInQkok7KsL/fqV6EuX2sIQ7KfIiuOc3vs/n4FbMqFcFbhwnP9CGd+LiAI5D5qehJK697cpGSd2wu458M1JuPk25z4cJahUFobIyEgOHDhQJouDqnLgwAEiCzOk9LZtsHEitFToOA4ia/k9n/Gx6D7QHJhf9Bu2mAC05R3AA98rDB9e4i9fKq9Kio6OJikpieTkZLejuCIyMjLXFVFn9NxjMDQLqnSCJrf7P5jxvaYDIem/sO5znFurm6DnyXKGwEgsD606OTdvKmGlsjCEh4ef6lVsziAxEZgMFULgkndASuXBY+lXu7tzOePJ5U47Q2io24lMce2dByk7YBbwJ//dpa0g9mlQVv37T9Bd4byRUK2N22lMUYVXBmkGzdLh11/dTmN8IXEipEfAxsrwhz+4EsHvhUFEtonIahFZISLx3nk1ROQbEdnkfayeY/1HRWSziGwQkT7+zlcmrVsNDebCyarQ9V9upzHF1bgfNAEWzHE7iSmutGTY+RnMz4Trb4QKFVyJUVJHDJepamyOzhRjgHmq2hznfkRjAESkNTAMaAP0Bf4jInZs7GtTboIGQJcJEObOH57xoaYDIRTYYP0Zgt7W90Ez4NusEu+7kJNbp5IGAdmDBb0HDM4xf4qqnlTVrcBmoHPJxyvFls+GmFVwuCW0tkHySoWaF4EnFDJXQWam22lMUak6p5F2V4SodtCxo2tRSqIwKDBXRJaJyAjvvDqqugfA+1jbO78BsDPHtkneecYXVGHh7ZAFDJrmdhrjK2HlIaSltTMEu+RFcHQ9zDwBd5bcgHn5KYmrkrqp6m4RqQ18IyLrC1g3v99Ens4I3gIzAqBRo0a+SVkW/PA81N4Lv/WBBm3dTmN8KaY/sBYWfgWdOrmdxhRF4kTIDIcVAjNudDWK348YVHW393EfMAPn1NBeEakH4H3c5109CWiYY/NoYHc++3xTVeNUNS77pjXmLNIPw+b/g+2hcNuHbqcxvtbMO8rupi/czWGKJv0I7JgKSwT6DYEaNVyN49fCICIVRaRy9nPgSiAB+ALIHkP2ViC71ewLYJiIRIhIDE6fzl/8mbHM+PYuKHcSQu6EGjXdTmN8LaoTZIWBJ8HaGYLR9o8gKxXmpLva6JzN36eS6gAzxDlXFgZ8pKpzRGQpMFVE7gB2ANcBqOoaEZkKrAUygZGqaqODFdf+n+HQNJgfAS8+53Ya4w8h4RDWFpqvgOXLobNdsxFUNk+EA5VAo6BXL7fT+LcwqOoWoH0+8w8Al59hm2eAZ/yZq0zxZML8m+EQ0PSRErmRuHFJ04GgK2DhF1YYgsnB5XBoOcwEhv8VQtzvd+x+AuNfG16B9E0wozLc9xe30xh/ym5n2DLT3Rzm3CROBE8Y/ATcdpvbaQArDKXbiR2w4nFYDvR9HCpXdjuR8adq7SEjAlgLGRlupzGFkZkC2ybBinLQvS80bHj2bUqAFYbSLP5+SE+HL6PgTyPdTmP8LSQUyrWHFplOO4MJfDumQcZR+ColIBqds1lhKK12fga7voBPPHD3Y1CxotuJTElocY3TXfSH6W4nMYWR+D84Vgn2R8HAgW6nOcUKQ2mUcQyW3Q8HKsKKunDPPW4nMj6SmJjI999/T9aZbuWZ3c6wfVbJhTJFc2Q9JP8Is1LglluhXDm3E51ihaE0Wj0WUnbBqydgzGNQvrzbiUwxeDweZs+ezdVXX03z5s3p1asXTZs25bnnnmP//v25V65yPqRXgJB11s4Q6La8BRoC8z0BdRoJrDCUPodWOFciJdSGtGhnzBUTlA4dOsT48eNp0aIF/fr1Y/ny5TzxxBNMmTKFpk2bMmbMGKKjo7ntttuIj493NhKByA7QIguy55nAk5UOW96D9RWhdVdo3drtRLlYYShNPFnwy91AZXhlLzz+OBTm3s8moKxatYq7776b6OhoRo8eTb169ZgyZQrbt29n7NixDB06lHnz5pGQkMAdd9zBtGnT6NSpE127duXDDz/kZMxAqA4s+sTtt2LOZNcXcDIZZh4LzC9vqhrUU8eOHdV4bXhddRKqNzVRbdxY9eRJtxOZQkpPT9epU6dqjx49FNDy5cvrnXfeqb/++utZtz18+LC++uqr2qJFCwW0Vs0ofWwQuvP2Zv4Pbormuz6qb1VUrVRB9ehRVyIA8XqGz1XXP9iLO1lh8ErZrTq1iuqU9s4/68SJbicyhbBnzx598skntX79+gpoTEyMjhs3Tg8cOHDO+8rKytK5c+fqwIEDVQQNFfQP11yj33//vXo8Hj+kN0VybKvqJFEdFq46fLhrMawwlAU/DFWdHKHas5Vq06aq6eluJzJn4PF4dNGiRXrDDTdoeHi4Atq3b1/98ssvNTMz0yevseWVi/SvV6I1qlRRQNu0aaMTJkzQY8eO+WT/phhWPqH6oajWRHXRItdiFFQYrI2hNNj9Nez4GEKugfnr4IknIDzc7VTmd1JTU3n77bfp2LEj3bp1Y9asWYwcOZKNGzeeuuooNNQ3d7KN6Xkzz90KSY/ewNtvv01ERAT33nsvDRo0YNSoUWzcuNEnr2POkScLtrwNO6pAzfPhoovcTpS/M1WMYJnK/BFDRorq501Uv2ip2r61aosWqhkZbqcyOWzZskUffvhhrVGjhgLatm1bfeONN/z77f3ELqe9aVRzVXWOUhYvXqw33njjqaOUPn366MyZM312lGIKIekr59+lE6rjxrkaBTtiKMXWPAPHt8CJ62HlWhg7FsJK4sZ8piAej4e5c+cycOBAmjZtyksvvUSvXr2YP3/+qauOKlWq5L8AFepDanWITIT0dETk1FVLO3fu5Omnn2b16tUMGDCA5s2bM27cOA4ePOi/PMaROBEyysOqULj5ZrfTnNmZKkawTGX6iOHwWtXJ4aqLblZt1Uq1dWtV+/bnqsOHD+srr7xy6gqh2rVr6+OPP647d+4s+TBT+qq+heqPC/JdnJ6erp988oleeumlCmhkZKQOHz5cly9fXsJBy4iUPaofhakOL686ZIjbaeyIoVRShaX3QFgl2NoV1q1zjhZ8dI7anJuEhIRT5/AfeOABoqKimDRpEjt27ODpp58mOjq65EO1vR4igSWT8l0cHh7Otddee+oo5tZbb2XKlCl06NCBbt26MXnyZNLT00s2c2m29T3QTJidGnA9nfM4U8UIlqnMHjFsfts5V7nhDdXmzVXbtVPNynI7VZmSkZGh06ZN0549eyqgERERevvtt2t8fLzb0Rxp+52/kYeaFHqTQ4cO6fjx47VZs2YKaJ06dfSJJ57QXbt2+TFoGeDxqH7eTPWl6qoNGgTEkT12xFDKpO2HFQ9DrW6wuBxs2gRPPhkQd34qC/bt28czzzxDTEwM1157LVu3buW5554jKSnp1FVHASEiCk7UhPLbnOHXC6FatWqMGjWKDRs2MHv2bOLi4nj66adp3LgxQ4cO5YcffsD5TDHnZN9COL4ZZhyG228P/CP7M1WMYJnK5BHD4tucc5XJv6o2aaLaoYPzjcT4jcfj0SVLluhNN92k5cqVU0CvuOIK/fzzzwP7qp6PB6i+g+rCeUXexebNm/Whhx7SatWqKaDt2rXTN998U48fP+7DoKXcohtV349ULYdqYqLbaVTVjhhKl70LYMu70Oov8Fk8bNkCTz3lDJ5mfC4tLY333nuPzp0707VrVz7//HPuvvtu1q1bd+qqI1/1PfCLdjdAOeCXD4u8i6ZNmzJu3Dh27drFxIkTERFGjBhBdHQ0Dz30EJs3b/Zd3tIo/ZBzQ56l4XBJL2jSxO1EZ3emihEsU5k6Ysg8qTrzfNXPYlRPHFJt1Ei1c2c7WvCDbdu26ZgxYzQqKkoBbdWqlb7++ut61KVxbYos/YjqB6g+fJ7PdunxePTHH3/UYcOGaVhYmIqI9uvXT2fNmqVZ1s6V1/p/O209jVH96CO305xCMA2JAfQFNgCbgTFnW79MFYbV/3D+wHbNUp0wwfnnmzPH7VSlhsfj0W+++UYHDRqkISEhGhISotdcc43OmzcvuMca+m9d1SdDVNPSfL7r3bt369ixY7Vu3boKaNOmTfXFF1/UgwcP+vy1gpLHo/pVO9XXq6tWr66amup2olMKKgziLA8MIhIKbASuAJKApcANqrr2TNvExcVpfBHGnc+c+28yE6ZCrbpQpx7UrAthERASDhLuPOZ6Xs77c9jpZafm5Vg/11TOWd8Xp3mObYE5HaD+1dDxLWjTBho3hnnz7DRSMaWkpPDRRx/x+uuvs379emrWrMldd93FPffcQ6NGjdyOV3xTr4WTn0Ljr6HHlX55ifT0dGbMmMFrr73Gjz/+SPny5bnpppsYOXIkLVu29MtrBoUD8fBNd5gUCs3ugvHjfbr7sLAwworYoVVElqlqXL7LAqwwXASMVdU+3p8fBVDVZ8+0TVELwwtDWrG2atc889esWcPSpUsJDw/nxhtvzLN8xYoVrFixggoVKnD99dfnWb506VLWrFlDlSpVGDJkSJ7lP/30Exs3biQqKooBAwbkWb5w4UK2bNlC3bp16du3b57l8+bNY+fOnTRs2JDLL788z/I5c+bw22+/0aRJE3r06JFn+cyZMzlw4AAtWrTg4osvzrN8+vTpHD16lDZt2tCpU6c8y6dOnUpKSgqxsbHExsbmWT5p0iQyMjLo1KkTbdq0ybP83XffBeDiiy+mRYsWuZZlZGQwaZJzzX2PHj1o8rtzsSkpKUydOhWAyy+/nIYNG+ZafvToUaZPd+513LdvX+rWrZtr+YEDB5g5cyYAAwYMICoqKtdyVaVXr15cf/31zJo1i6NHj+ZaHh0dTe/evXP9HnKKiYnh0ksvzfV7yCnn7zz795BT9u885+8hp+zfec7fQ05xcXG0bduWI0eOMGPGjNMLUvfDsTVcpOVo+ch/2b9/P19++WWe7bN/57/99htz5szJszz7d75z507mzZuXZ3n273zu3LnMnz+fvXv34vF4Ti23v70z/+3l/J0PGTKEKlWq5Fqe83d+/fXXU6FCBQC6dOnCPUW8dW9BhSHQxk5oAOzM8XMS0OX3K4nICGAEUORvdD0eeZfUOXOcy/gy0k89NqsUwZDyoOUgIxQIEWeIifAwCA+n2cUXMPTSlmj5SDLCa3v3ps6k0Kx2R0L7nIdqKBkVqnuXeddRpdnVLQntXRWPRJJZpWLu7YHmfesSkpaJJ6wqmdXL5VpGuRrcc+edhLzwAp569ciMicnzvu6//35CQkLweDxkZmbmWT569GhCQkLIysrK977BY8aMQUTOuPzxxx8vcPmTTz5Z4PJnn3VqfGZmZq4PjcIsFxEuvPDCApdnf6Dkt7xJkyanPpAyMjLI+aWoevXqNG/ePN9iHPQia8BR4GhCibxcs2bN2L17N02aNGHfvn2n/g7L3N+eKqRsh5NCk9QKXOzd/+//9sBp4M/+UnGm5dlfSnIu79Ilz8ejTwTaEcN1QB9VvdP7881AZ1W9/0zbFPWIoUBHjsCaNZCQcPoxIQH27Tu9TrVq0LatM7Vpc/p5zZq+zfJ7L70EDz0ECxZAPkcExuTrjYZwaBeMToWICLfTlA2Jb8PPd8CTwKj/wL33up0ol2A6YkgCch6jRQO7SzxF1apw8cXOlFNycu5CsWYNTJkChw+fXqdOndyFIrtw/O7QsEhOnIB//Qt697aiYM5NVHeoNBkWfwM9+7udpmxInAgp1WBHKtxwg9tpzkmgFYalQHMRiQF2AcOAP7obKYdataBnT2fKpgq7d+c9upg4EXKeg27YMO8RRqtW4D1XWCivv+4Up6ee8tU7MmVFx5thyWRY9qEVhpJweA3sXwyzI+Ha65wzDEEkoAqDqmaKyH3A10Ao8LaqrnE5VsFEoEEDZ+rT5/R8jwe2bz9dKLILx7x5p4cnEHE6u/z+6KJlSyhXLvfrHDsGzz8PV10VuDf3MIGr8eWwMAQO/uh2krIhcSJoKHyTBl8E+IB5+QiowgCgqrOAWW7nKLaQEIiJcaacVx9lZsLmzbmPLhIS4MsvIbvBLCwMWrTIfXTx009w4IAzJpIx5yq0HKQ2gurbIS0NIiPdTlR6ZZ2Ere/DlhpQpyp4G5WDScAVhlIvLAzOP9+Z/vCH0/NPnoQNG3IfXcTHQ87LEgcMgHwu4zOmUGr1hKrvwuLZcNk1bqcpvXbOgPSD8Akw/MGg7GdkhSFQRERAu3bOlNOJE869FtavdxqdjSmqTrfBonfh1w+tMPhT4kRIrwprj8Ktt7qdpkisMAS6ihUhLs6ZjCmOhpdAaigc/sntJKXX8S2wdx58Vwmuuhrq13c7UZHY6KrGlBUhoZAWA1G/Oe0MxvcS3wJC4KvjgX+XtgJYYTCmLKnbC2oBP84466rmHHkyYcs7sLsWhNeBq692O1GRWWEwpizpfLvzuGqyuzlKo92zIXUPTE122hbCw91OVGRWGIwpS6K7wIkwOPaz20lKn8SJkFUJlntg+HC30xSLFQZjyhIRONkMau2D1FS305QeKbth91ewOBwuusTppBrErDAYU9bU7w3VgB8+djtJ6bH1XdAsmH4I7rzT7TTFZoXBmLLmIu8H15q893QwRaAe52qkg3UhpTJce63biYrN+jEYU9bUbQdHysGJpW4nKR32fu/0X/i0HNxwm9P3KMjZEYMxZY0IZLaA2vsh5YTbaYJf4kTQCrAoPaj7LuRkhcGYsqhhH6gE/PCh20mC28kDsHM6rKgELduWmrHMrDAYUxZltzOsm+ZujmC39QPwpMPH+5xG5yAcMC8/VhiMKYvqnA8HIyF1mdtJgpeqcxrpeB3YWw5uusntRD5jhcGYsspzPtQ+BMePuJ0kOB34GY6sgZnHYPBgiIpyO5HPWGEwpqxq3A/KAz+873aS4LT5f0AEfJtSahqds1lhMKas6uZtZ9gw3d0cwSjjKGyfAhujoE7jUnevFCsMxpRVNWMguQKk/+p2kuCz/WPISoFJu+H2251b+ZYipevdGGPOUWuocwSOHXQ7SHDZ/D84WRsScQpDKeO3wiAiY0Vkl4is8E79cix7VEQ2i8gGEemTY35HEVntXfaqSCm59suYQBXTH8KBH952O0nwOLQSDi6FuelwxZXQqJHbiXzO30cM41U11jvNAhCR1sAwoA3QF/iPiIR6158AjACae6e+fs5nTNl2yZ2QBWz6zO0kwSPxLSAcvjxc6hqds7lxKmkQMEVVT6rqVmAz0FlE6gFVVHWxqirwPjDYhXzGlB01GkByRchc5XaS4JCZ6nRq21kXIqJg0CC3E/mFvwvDfSKySkTeFpHq3nkNgJ051knyzmvgff77+XmIyAgRiReR+OTkZH/kNqbsCGkHtY7B4d/cThL4dk6HjMPw0W64+WaIiHA7kV8UqzCIyLcikpDPNAjntFBTIBbYA7yYvVk+u9IC5uedqfqmqsapalytWrWK8xaMMU0HOOMs//iW20kCX+JEyIqC1Vml9jQSFLMwqGpvVW2bz/S5qu5V1SxV9QD/Azp7N0sCGubYTTSw2zs/Op/5xhh/6n4HZABbZrqdJLAd3QT75sPCUOjUGdq2dTuR3/jzqqR6OX68BkjwPv8CGCYiESISg9PI/Iuq7gGOiUhX79VItwCf+yufMcarWm3YWwU8CWdftyzb8hYQCtP2lYq7tBXEnzfqeV5EYnFOB20D7gZQ1TUiMhVYC2QCI1U1y7vNvcC7OB31Z3snY4y/hbeHmj/AwR1Qo/RdfllsngzY8i7sj4b0ZBg61O1EfuW3wqCqNxew7BngmXzmxwOl9/jMmEDV4ho48gMsegsGPOl2msCz60tI2wsfl4frh0KVKm4n8ivr+WyMge63Qhqw7Su3kwSmxImg1WBJaqludM5mhcEYA1VqwN5qIGvdThJ4TuyEPXNgWSVo3hK6dXM7kd9ZYTDGOCI6QI1U2L/Z7SSBZcs7oB74MMk5WigDI/VYYTDGOM4f4jz+NNHdHIHEk+UMgXGsMRwKg1tucTtRibDCYIxx9LgZTgA77GLAU/bOg5QdMP0Q9O8Pdeq4nahE+PNyVWNMMKlUBfbWgKgNbicJHJv/B1SG747CjNLf6JzNjhiMMaeVj4OqJ2HvareTuC9tH+z6HNbXhNr1oW/ZGezZCoMx5rQ21zmPS+z+DGz9wOnY9s42uO02CCs7J1isMBhjTut+AxwBkua6ncRdqpD4P0hrBEkKw4e7nahEWWEwxpxWsSLsqwnhm5wPx7IqeREc3QCzU6FnT2ja1O1EJcoKgzEmt0qdoVIG7FnmdhL3JE4EqQAzk8tET+ffs8JgjMmt7TDn8ed33M3hlvTDsGMqbK8PkVXhD39wO1GJs8JgjMntkiGwH9jzrdtJ3LF9MmSlwrs74MYboXx5txOVOCsMxpjcKlaE/XUgItEZCqKs2TwRshrAhvQyeRoJrDAYY/JTpSuUz4Kkn9xOUrIOLodDy+F7gdhY6NDB7USusMJgjMmr/Q3O49L33M1R0hIngkTAx0ml/i5tBbHCYIzJ65KBsAfY+53bSUpOZgpsmwR7G0NWBPzxj24nck3Z6cpnjCm88uXhYD2I2ub0/g0JdzuR/+2YChlHYZLHuRKpenW3E7nGjhiMMfmr1g3KeWDHAreT+N/Jg7DyMaARxB8vs43O2awwGGPyd6H3VMqy993NURLi73MGzZtRG2JinN7OZVixCoOIXCcia0TEIyJxv1v2qIhsFpENItInx/yOIrLau+xVEed2SCISISIfe+f/LCLnFSebMaaYLrkKdgokl/Ijhu1Tnb4LDe6HafHO0UJI2f7OXNx3nwAMARbmnCkirYFhQBugL/AfEQn1Lp4AjACae6fssWzvAA6pajNgPPBcMbMZY4ojMhIO14cKOyErze00/pG6B5beC1GdYXqmUxBuu83tVK4rVmFQ1XWqmt9dPQYBU1T1pKpuBTYDnUWkHlBFVRerqgLvA4NzbJN9bdw04PLsowljjEtqdIcwhW2lsBe0Kvx8p9PLOfQ+ePU1uOsuaNDA7WSu89fxUgNgZ46fk7zzGnif/35+rm1UNRNn8N+o/HYuIiNEJF5E4pOTk30c3RhzStxN4AF+/dDtJL6XOBF2z4KWY+GOx50RVMeNcztVQDhrYRCRb0UkIZ9pUEGb5TNPC5hf0DZ5Z6q+qapxqhpXq1atgt+AMaboLroctgoc/MHtJL51fAssfxDqXA4vroJdu+DDD6FSJbeTBYSz9mNQ1d5F2G8S0DDHz9HAbu/86Hzm59wmSUTCgKrAwSK8tjHGVyIj4WhDOG8HZByH8FLwwenJgsW3goTB/iHw4UgYOxa6dHE7WcDw16mkL4Bh3iuNYnAamX9R1T3AMRHp6m0/uAX4PMc2t3qfXwt8522HMMa4qdalEApsne12Et/YMB6Sf4SmT8K9jzsF4bHH3E4VUIp7ueo1IpIEXAR8JSJfA6jqGmAqsBaYA4xU1SzvZvcCE3EapBOB7L+2t4AoEdkMjAbGFCebMcZHOt8EGcDKj9xOUnyHE5yObNGD4ZGZcPIkfPBBmbqfc2EU67ehqjOAGWdY9gzwTD7z44G2+cxPA64rTh5jjB907QHfCTQM8pFWs9Jh8c1QrhqsiINvH4f//heaN3c7WcCxMmmMKVhkJJxoDBW2OUNHRNRwO1HRJDwFh1ZA41fgj3+F/v2dy1NNHmW7e58xpnDq9HKuG9zypdtJimb/Elj7LDS+BUa+DVWrwltvgXWVypcVBmPM2XW9EdKA1R+7neTcZabA4lugfDR8WR1WrnSKQu3abicLWFYYjDFn16UbbAqBY0vcTnLuVjwCxzZBhQfhuVdhxAjnNJI5IysMxpizi4iA1BgofxBSf3M7TeHt+QY2vgYx98Jd453ezS++6HaqgGeFwRhTOPWvcB4TZ7qbo7DSD8OS26HK+fDfw9a7+RxYYTDGFM5Fw+AEsGaq20kKJ/7PkPYbnLgF3psMjz9uvZsLyQqDMaZwOneFjSFwYqnbSc5ux6ew7QNo9Ge49wXr3XyOrDAYYwonIgJONoPII3B8m9tpziz1N1h6N1TvCE+uPN27ObwM3LfaR6wwGGMKL9p7M8YtAdrOoAq/jHAG/Eu8Ar75DsaPt97N58gKgzGm8C6+3rlTyrpP3U6Svy3vwq6ZUHcU/GW89W4uIisMxpjC69wZNoRCarzz7TyQHN8Gyx6AWj1g1GyoUgUmTrTezUVghcEYU3jlykFGcyh3Ao5tdDvNaeqBJbc5zxe0gpWrnN7Ndeq4GitYWWEwxpybxv2cx8Qv3M2R04ZXYN8CqPQn+OebTu/mAQPcThW0rDAYY85NtyGwH9iQ74j7Je/IWljxKNS5Cu79yHo3+4ANu22MOTedOsHkUKj6q3MKR1z8funJcAbIC68MkyvCrt2waJH1bi4mO2IwxpybcuUgqxWEp8Hh1e5mSXgGDi6DjFvhrWnWu9lHrDAYY85dTAC0MxxYCmv+AXX+AH9627liyno3+4QVBmPMues+GPYAmz935/UzU51TSJH14KX9Tu/mDz+03s0+YoXBGHPu4uJgYxikrwJPZsm//sq/wdH1sGcAzFpgvZt9rFiFQUSuE5E1IuIRkbgc888TkVQRWeGd3sixrKOIrBaRzSLyqojT+0REIkTkY+/8n0XkvOJkM8b4UXg40AbCMpxz/CVp7/ew4WWo+UcY/bb1bvaD4h4xJABDgIX5LEtU1VjvdE+O+ROAEUBz79TXO/8O4JCqNgPGA88VM5sxxp+aeu+CVpLtDOlHYPFtUKk5PJZgvZv9pFiFQVXXqeqGwq4vIvWAKqq6WFUVeB8Y7F08CHjP+3wacHn20YQxJgD16A/bKdkB9ZaPgtQk+LULxFvvZn/xZxtDjIj8KiILRKS7d14DICnHOkneednLdgKoaibOUF1R+e1YREaISLyIxCcnJ/snvTGmYB07wqYwyFwLWWn+f72kz51B8irdCE9Ost7NfnTWwiAi34pIQj7ToAI22wM0UtULgdHARyJSBcjvCCB7JK6CluWeqfqmqsapalytWrXO9haMMf4QHg6h7SA0C/Yv8e9rpe2Dn++CKu3ggfnWu9nPztrzWVV7n+tOVfUkcNL7fJmIJAItcI4QonOsGg3s9j5PAhoCSSISBlQFDp7raxtjSlDzgeBZ7pxOqtPTP6+hCr/cAxlHYV5X2LHGejf7mV9OJYlILREJ9T5vgtPIvEVV9wDHRKSrt/3gFiD7QugvgFu9z68FvvO2QxhjAtWlfWELsO0r/73G1g8gaQZwHUyYab2bS0BxL1e9RkSSgIuAr0Tka++iHsAqEVmJ05B8j6pmf/u/F5gIbAYSgdne+W8BUSKyGef005jiZDPGlICOHWFTOGRtcu6a5msndsCy+6FqFxj5pfVuLiHFGkRPVWcAeYZYVNVPgXxv8aSq8UDbfOanAdcVJ48xpoSFhUF4LIQsheQfoX7fs25SaOqBJcOdx4mhkJZuvZtLiPV8NsYUT6uBkInvL1vd+DrsnQeHrobPfrLezSXICoMxpnguvRI2ATvm+G6fR9bDir9C5R7wwAzr3VzCrDAYY4qnQwfYXA50C6QfKv7+PJnOAHmhFeC5A1ClqvVuLmFWGIwxxRMWBhEdnJ5IexcUf39rnoWDS2FTd1i0xno3u8AKgzGm+NoMdHoubf2yePs5uBwSnoLyveGxL5zTR9a7ucRZYTDGFF/P3rABSPr6rKueUVYaLL4ZytWCMeud3s0vveSziKbwrDAYY4rvwgshMQJIgtS9RdvHysfhyFr4sTUk7nEuTbXeza6wwmCMKb6wMCjvvSXL3u/Pffu9C2D9SxByJbw8z3o3u8wKgzHGN9oNgBOceztDxjFYchtENoJRv1jv5gBghcEY4xuX9Yb1wJ5vz2275aMhZQd8WguOWe/mQGCFwRjjG7Gx3naGvXBie+G22fUlJE6EE73go3insdl6N7vOCoMxxjdCQ6GSt12gMO0Mafvh5zshsgWMWuj0bh4xwr8ZTaFYYTDG+M6F/Z17L56tnUEVlt4LJw/CGwIVrHdzILHCYIzxncsuh3XA3u+cD/8z2T4Zdk6D3RfDNxusd3OAscJgjPGd9u1hSyRwCI5tyn+dlCRYOhLKtYW/LrDezQHICoMxxndCQ6HKRc7zvd/lXa7q3GPBkwHPHIAmzax3cwCywmCM8a24q+EA+d/uc9ME+O0bWNUWVu+DDz6w3s0ByAqDMca3LusFa4B9C5y7r2U7ugl+fRikPfzzZ6d3c9eursU0Z2aFwRjjW+3awbYKIMfgcIIzz5MJS24FCYe/bbPezQHOCoMxxrdCQ6FaN+f53nnO47oXYP9imNcI9mVY7+YAV6zCICIviMh6EVklIjNEpFqOZY+KyGYR2SAifXLM7ygiq73LXhVxLlwWkQgR+dg7/2cROa842YwxLuraD/YA22bBoRWw+v8g7UJ4c7X1bg4CxT1i+AZoq6rtgI3AowAi0hoYBrQB+gL/EZFQ7zYTgBFAc+/U1zv/DuCQqjYDxgPPFTObMcYtPXvCWuDAj85tOkOqwsNrrHdzkChWYVDVuaqa6f1xCRDtfT4ImKKqJ1V1K7AZ6Cwi9YAqqrpYVRV4HxicY5v3vM+nAZdnH00YY4JMu3awrSJIGhxeDVOqQKj1bg4WvmxjGA7M9j5vAOzMsSzJO6+B9/nv5+faxltsjgBR+b2QiIwQkXgRiU9OTvbZGzDG+EhICNTsDpnAgfYwY4v1bg4iYWdbQUS+Berms+gxVf3cu85jOH8Ck7I3y2d9LWB+Qdvknan6JvAmQFxcXAH97o0xrrm4L/xlDhxYab2bg8xZC4Oq9i5ouYjcCvQHLveeHgLnSKBhjtWigd3e+dH5zM+5TZKIhAFVgYOFeA/GmEDUsyeMAppZ7+ZgU9yrkvoCjwADVTUlx6IvgGHeK41icBqZf1HVPcAxEenqbT+4Bfg8xza3ep9fC3yXo9AYY4LNBRfAmDEwbZr1bg4yZz1iOIvXgAjgG2878RJVvUdV14jIVJzrEjKBkaqa5d3mXuBdoDxOm0R2u8RbwAcishnnSGFYMbMZY9wUEgLPPut2ClMEEuxfyuPi4jQ+Pt7tGMYYE1REZJmqxuW3zHo+G2OMycUKgzHGmFysMBhjjMnFCoMxxphcrDAYY4zJxQqDMcaYXKwwGGOMySXo+zGISDKwvYib1wT2+zCOvwVT3mDKCsGVN5iyQnDlDaasULy8jVW1Vn4Lgr4wFIeIxJ+pg0cgCqa8wZQVgitvMGWF4MobTFnBf3ntVJIxxphcrDAYY4zJpawXhjfdDnCOgilvMGWF4MobTFkhuPIGU1bwU94y3cZgjDEmr7J+xGCMMeZ3rDAYY4zJpUwWBhF5W0T2iUiC21nORkQaisj3IrJORNaIyANuZyqIiESKyC8istKb90m3M52NiISKyK8i8qXbWc5GRLaJyGoRWSEiAX0jEhGpJiLTRGS99+/3IrcznYmItPT+TrOnoyIyyu1cZyIiD3r/fyWIyGQRifTp/stiG4OI9ACOA++ralu38xREROoB9VR1uYhUBpYBg1V1rcvR8uW9ZWtFVT0uIuHAj8ADqrrE5WhnJCKjgTigiqr2dztPQURkGxCnqgHfCUtE3gN+UNWJIlIOqKCqh12OdVYiEgrsArqoalE7z/qNiDTA+X/VWlVTvXfLnKWq7/rqNcrkEYOqLsS5fWjAU9U9qrrc+/wYsA5o4G6qM1PHce+P4d4pYL99iEg0cDUw0e0spYmIVAF64NyyF1VND4ai4HU5kBiIRSGHMKC8iIQBFYDdvtx5mSwMwUpEzgMuBH52OUqBvKdmVgD7gG9UNZDzvgz8FfC4nKOwFJgrIstEZITbYQrQBEgG3vGeppsoIhXdDlVIw4DJboc4E1XdBYwDdgB7gCOqOteXr2GFIUiISCXgU2CUqh51O09BVDVLVWOBaKCziATk6ToR6Q/sU9Vlbmc5B91UtQNwFTDSe1o0EIUBHYAJqnohcAIY426ks/Oe8hoIfOJ2ljMRkerAICAGqA9UFJGbfPkaVhiCgPdc/afAJFWd7naewvKeOpgP9HU3yRl1AwZ6z9tPAXqJyIfuRiqYqu72Pu4DZgCd3U10RklAUo6jxWk4hSLQXQUsV9W9bgcpQG9gq6omq2oGMB242JcvYIUhwHkbc98C1qnqS27nORsRqSUi1bzPy+P8Ea93NdQZqOqjqhqtqufhnD74TlV9+s3Ll0SkovcCBLynZa4EAvLKOlX9DdgpIi29sy4HAvKCid+5gQA+jeS1A+gqIhW8nw+X47Q9+kyZLAwiMhlYDLQUkSQRucPtTAXoBtyM8202+1K6fm6HKkA94HsRWQUsxWljCPjLQINEHeBHEVkJ/AJ8papzXM5UkPuBSd6/hVjgn+7GKZiIVACuwPkGHrC8R2HTgOXAapzPcZ8OjVEmL1c1xhhzZmXyiMEYY8yZWWEwxhiTixUGY4wxuVhhMMYYk4sVBmOMMblYYTDGGJOLFQZjjDG5WGEwxodE5K8i8mfv8/Ei8p33+eWBPtyGMdmsMBjjWwuB7t7ncUAl71hXlwA/uJbKmHNghcEY31oGdPSOaXQSZ+iVOJxiYYXBBIUwtwMYU5qoaoZ3tNbbgZ+AVcBlQFN8PNCZMf5iRwzG+N5C4C/exx+Ae4AVagOTmSBhhcEY3/sBZ5TZxd5x/dOw00gmiNjoqsYYY3KxIwZjjDG5WGEwxhiTixUGY4wxuVhhMMYYk4sVBmOMMblYYTDGGJOLFQZjjDG5/D/8iLOj4PiLnQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#将系数进行绘图\n",
    "plt.plot(range(1,9),(reg.coef_*100).tolist(),color=\"red\",label=\"LR\")\n",
    "plt.plot(range(1,9),(Ridge_.coef_*100).tolist(),color=\"orange\",label=\"Ridge\")\n",
    "plt.plot(range(1,9),(lasso_.coef_*100).tolist(),color=\"k\",label=\"Lasso\")\n",
    "plt.plot(range(1,9),[0]*8,color=\"grey\",linestyle=\"--\")\n",
    "plt.xlabel('w') #横坐标是每一个特征所对应的系数\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可见，比起岭回归，Lasso所带的L1正则项对于系数的惩罚要重得多，并且它会将系数压缩至0，因此可以被用来做特征选择。也因此，我们往往让Lasso的正则化系数$\\alpha$在很小的空间中变动，以此来寻找最佳的正则化系数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了解决多重共线性这个核心问题之外，线性模型还有更重要的事情要做：提升模型表现。这才是机器学习最核心的需求，而Lasso和岭回归不是为此而设计的。下面，让我们来认识一下为了提升模型表现而做出的改进：多项式回归。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.3 非线性问题：多项式回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在回归中，线性数据可以使用如下的方程来进行拟合："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "y = w_0+w_1x_1 + w_2x_2 + w_3x_3....w_nx_n\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也就是我们的线性回归的方程。根据线性回归的方程，我们可以拟合出一组参数$\\boldsymbol{w}$，在这一组固定的参数下我们可以建立一个模型，而这个模型就被我们称之为是**线性回归模型**。所以建模的过程就是寻找参数的过程。此时此刻我们建立的线性回归模型，是一个**用于拟合线性数据的线性模型**。作为线性模型的典型代表，我们可以从线性回归的方程中总结出线性模型的特点：**其自变量都是一次项**。\n",
    "\n",
    "那线性回归在非线性数据上的表现如何呢？我们来建立一个明显是非线性的数据集，并观察线性回归在拟合非线性数据集时的表现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:36:40.320361Z",
     "start_time": "2024-03-06T03:36:40.313653Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:37:20.320828Z",
     "start_time": "2024-03-06T03:37:20.213805Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAyfUlEQVR4nO3df3Ac5X3H8c9x2LIdLFlGtSwPwnawJFxIiLFbsAcHDEJ3UupimjJgzbhmSpiBhmSMhzYxTInTGaK2IZROCQY6DD8KEWTimDCDdLI19Q8y2AVTGSgErWxM5WK7Bp+QDO1IQWz/oFJ81ul0q9u9fXb3/ZrZGd/e3t2jvfPt957n+3yfmG3btgAAAALiLL8bAAAA4ATBCwAACBSCFwAAECgELwAAIFAIXgAAQKAQvAAAgEAheAEAAIFC8AIAAALlbL8b4LbPP/9cR48e1cyZMxWLxfxuDgAAyINt2zp16pTmzZuns87K3bcSuuDl6NGjqq6u9rsZAABgEo4cOaLzzjsv5zGhC15mzpwp6Ys/vrS01OfWAACAfAwMDKi6unr0Op5L6IKXkaGi0tJSghcAAAImn5QPEnYBAECgELwAAIBAIXgBAACBQvACAAACheAFAAAECsELAAAIFIIXAAAQKAQvAAAgUAheAABAoISuwi4AAIWyLEuHDh3SokWLVFNT43dzcAZPe1727Nmj1atXa968eYrFYnrhhRdyHr9r1y7FYrEx27vvvutlMwEAkCSl02klk0nV1dWpqalJtbW1SiaT6uvr87tpOI2nwcunn36qSy65RA899JCjx3V3d+vYsWOjG1EvAKAYmpub1dnZmbGvs7NTa9eu9alFyMbTYaPGxkY1NjY6ftycOXM0a9Ys9xsEAMA4LMtSR0fHmP3Dw8Pq6OhQT08PP6YNYWTC7pIlS1RVVaVrrrlGO3fuzHns4OCgBgYGMjYAAJw6dOhQzvsPHjxYpJZgIkYFL1VVVXrssce0detW/fKXv1RdXZ2uueYa7dmzZ9zHtLS0qKysbHSrrq4uYosBAGFxwQUX5Lx/0aJFRWoJJhKzbdsuygvFYtq2bZvWrFnj6HGrV69WLBbTiy++mPX+wcFBDQ4Ojt4eGBhQdXW1+vv7VVpaWkiTAQARk0wm1dnZqeHh4dF98Xhc9fX1SqVSPrYs/AYGBlRWVpbX9duonpdsLr/8cvX09Ix7f0lJiUpLSzM2AAAmo7W1VfX19Rn76uvr1dra6lOLkI3xdV66urpUVVXldzMAABFQXl6uVCqlnp4eHTx4kDovhvI0ePnkk08yEpwOHz6sAwcOaPbs2Tr//PO1adMmffDBB3r66aclSQ8++KAWLFigiy66SENDQ3rmmWe0detWbd261ctmAgCQoaamhqDFYJ4GL/v379eqVatGb2/cuFGStH79ej355JM6duyYent7R+8fGhrSXXfdpQ8++EDTp0/XRRddpJdeeklNTU1eNhMAAARI0RJ2i8VJwg8AADBDqBJ2AQAATmd8wi4AYHwsIIgooucFAALGsiz9/Oc/18qVK1lAEJFEzwsABEQ6nVZzc3PW9Xek3y0gSDE1hB09LwAQENlWPD7d6QsIAmFG8AIAATCy4vHpZevHwwKCCDuCFwAIgIlWPD4dCwgi7Mh5AYAAmGjFY+l3Cwgy6whhR88LAARAbW2tEomE4vH4uMewgCCiguAFAAIi24rHK1eu1PPPPy/LspRKpVReXu5T64DiYdgIAAKCFY9/h+J80UbwAgABE+UVj7PVukkkEmptbaXXKUIYNgIABEa2WjcjxfkQHQQvAIBA6OjoyFrrhuJ80UPwAgAwWjqdVjKZVDKZzHkcxfmig+AFAGC0iZZFGEFxvuggYRcAYKyRZRFyoThf9BC8AAAmzespy/ksi0Bxvuhh2AgA4NhIHkpdXZ2amppUW1urZDKpvr4+V19nomURtm/fTnG+CCJ4AQA4Vqwpy+MtixCPx5VIJHTttde6+noIBoIXAIAjI3koxZqynG1ZBIaKoo2cFwCAIxPloRw8eNDV/BeWRcCZCF4AAI5MlIfi1ZTlKC+LgEwMGwEAHJkoD4UAA14jeAEAOEYeCvzEsBEARNhk67SQhwI/EbwAQASl02k1NzdnVK9NJBJqbW11VDOFPBT4gWEjAIigYtVpAbxA8AIAEVPsOi2A2wheACBi8qnTApiM4AUAIsavOi2AWwheACBiqNOCoCN4AYAIok4Lgoyp0gAQQdRpQZARvABAhFGnBUHEsBEAAAgUghcAABAoBC8AACBQCF4AAECgELwAAIBAIXgBAACBQvACAAACheAFAAAECkXqAMABy7J06NAhKtL6iPcA9LwAQB7S6bSSyaTq6urU1NSk2tpaJZNJ9fX1+d00T1mWpfb2dvX09PjdlMi+BxiL4AUA8tDc3KzOzs6MfZ2dnVq7dq1PLfKWiYFC1N4DjC9m27btdyPcNDAwoLKyMvX396u0tNTv5gAIAcuyVFdXl/P+sA1fJJNJdXZ2anh4eHRfPB5XfX29UqlU0dsTxfcgapxcvz3tedmzZ49Wr16tefPmKRaL6YUXXpjwMbt379bSpUs1bdo0ffnLX9YjjzziZRMBYEKHDh3Kef/BgweL1JLisCxLHR0dGYGLJA0PD6ujo8OXIaSovQfIzdPg5dNPP9Ull1yihx56KK/jDx8+rKamJq1cuVJdXV26++679d3vfldbt271spkAkNMFF1yQ8/5FixYVqSXFYWKgELX3ALl5OtuosbFRjY2NeR//yCOP6Pzzz9eDDz4oSVq8eLH279+v+++/X9/85jc9aiUA5FZbW6tEIjHuMIopwxVuzcIxMVAIynuA4jAqYXfv3r1qaGjI2JdIJLR//3799re/zfqYwcFBDQwMZGwA4LbW1lbV19dn7Kuvr1dra6tPLfodt5NrRwKFeDyesT8ejyuRSPgWKJj8HqC4jApejh8/rsrKyox9lZWV+uyzz/TRRx9lfUxLS4vKyspGt+rq6mI0FUDElJeXK5VKybIstbW1ybIspVIplZeX+900T2bhmBgomPweoLiMK1IXi8Uybo9Mhjpz/4hNmzZp48aNo7cHBgYIYAB4pqamxqghipHk2jOdnlw7mfaOBAo9PT06ePCgKwXh3BrWMu09QPEZFbzMnTtXx48fz9h34sQJnX322Tr33HOzPqakpEQlJSXFaB4AGCef5NqamppJBw5uBArpdFrNzc0ZQVYikVBrayu9JpgUo4aNli9frh07dmTs2759u5YtW6YpU6b41CoA8E6hFWwnSq6tqKhwnA/jdlVdisvBdbaHTp06ZXd1ddldXV22JPuBBx6wu7q67P/8z/+0bdu2v//979vr1q0bPf69996zZ8yYYd955532O++8Yz/++OP2lClT7F/84hd5v2Z/f78tye7v73f97wEAt5w8edJOJBK2pNEtkUjY6XTa8XMlEgk7Ho9nPFc8HrcTiUTO+7xs04ju7u6M5ztzsyxr0s9tuu7ubrutrS3Uf6ObnFy/PQ1edu7cmfXDun79etu2bXv9+vX2lVdemfGYXbt22UuWLLGnTp1qL1iwwN6yZYuj1yR4ARAEToKKiaTT6axBx6uvvuoocHCzTSPa2tpytqGtrW3Sz20qL4LAKHBy/WZ5AAAosolK3W/fvl3XXnut4+c9M7m2vb1dTU1N4x7f1tY2WovLq/L7USzrb9rSCkFhzPIAAICxJkqybWhomFSdlpqaGjU2No4GA06KzXlVVdfUmjFeMXFphTAieAGAIpsoqJDcSWh1Ejh4WVXXxJoxXjFxaYUwIngBgCIbL6g4nVu/1PMNHLzsIYlScTkTl1YohNszz9xC8AIAPsgWVGRT6C91J4GD1z0kZw5rhVGxh8m8Ci7cXnLCbSTsAoCPtm/frkQiMe79fiS0ullVN4r6+vq0du1aT4vyeV34z4+kYyfXb4IXAPCZ2xcKt8rwozBeBoFeBhd+zRBjthEABIhbwzWmd/VHjVfDZF7PaApC0jHBCwD4zK2E1qiX4Tc1udRtXgcXQUg6JngBgAK4ecEs5Jd6lOuLRK3HyevgIgi1eQheAGASTLtgBqGr3ytR63EqRnBhem0eEnYBYBJMKwEfxTL8UnT/7mLMaJKKO/PMyfX7bE9bAgAhNDJEc6bTh2iKfcEc+TU+XkAVxgu4lF+PUxj/9pE8Ka+Di5qaGiPPH8NGAOCQqUM0E3X1hzGhNQjJpV6KQuG/bAheAMAhUy+Y481asm3bqPwcNwUhuRTuI3gBAIdMv2Ce+Ws87AmtpieXwn0k7ALAJBQrYbJQUUpoZVmDYCNhFwDkbZn8YiVMFipKCa1uJJeytEIwELwACB2vF607namzMUaYmp9jmmJ+ZlA4cl4AhE7YczycMD0/xxR8ZoKF4AVAqES5TP54vExoDcP0az4zwUPwAiBUTK3BMsKPi71bCz+ezrTlEQph+mcGYxG8AAgVU3M8TLjYu1nQLEzDLKZ+ZjA+ghcAoWJqjkeYLvZhG2Yx9TOD8RG8AAgd04qWhe1iH8ZhFtM+M8iNqdIAQse0Gixhq7USxmEW0z4zyI3gBUBomVKDJWwX+zCvYG3KZwa5MWwEAB4LY04FwyzwE2sbAUARBGUtJKcYZoFbnFy/CV4AoIi42KOYgrRWEwszAoChyKlAMYR9rSZyXgAA8JEXVZfDVFcoG3peAADwgZu9I6cPD9m2nfGcI06vKxT03j96XgAAkeXnwpJu9I5kW3ZioscHsYjgmQheAACR4/daU25VXc4WAL3xxhs5HxO0ukLZELwAACLH75wQN5ZYyBUASdJZZ2Ve4oNcV+hMBC8AEAF+Do+YxoS1ptyoujxRAPS1r30t43aYigiSsAsAhvCiJkfYp8xOhglrTbmxxMJEAdBzzz0nSaGsK0TPCwD4zMv8C7+HR0xkylpThS6xkM+yEzU1NWpsbAxV4CJRYRdARJhcaTSZTI77CzyVSk36eS3LUl1dXc77TTsXxeLVOZ+MQqouh2nZCSfXb3peAISa37NKJuJl/oUbSaFhZdLCkoX0jpSXlyuVSsmyLLW1tcmyLKVSqcAFLk6R8wIg1HINmxT7F3Y2XuZfmDI8YqKRi35Y1pqK2rIT9LwACC0TZpVMxMsAI5+ciKgLa05I2BG8AAitIAybeB1gmDQ8AriFYSMAoRWUYZPW1tYxSZduBRhhGx4BJGYbAQg5k2aVTIQAwz0mzy5Ddsw2AoD/F6RhE/IvCmf67DK4g54XAJFAr0Y0BKmnDZmM63l5+OGHtXDhQk2bNk1Lly7Vyy+/PO6xu3btUiwWG7O9++67xWgqgJCiVyP8gjC7DO7wPHh5/vnntWHDBt1zzz3q6urSypUr1djYqN7e3pyP6+7u1rFjx0Y3vnAAALkEYXYZ3OF58PLAAw/olltu0be+9S0tXrxYDz74oKqrq7Vly5acj5szZ47mzp07up05jRAAgNMFZXYZCudp8DI0NKTXX39dDQ0NGfsbGhr0yiuv5HzskiVLVFVVpWuuuUY7d+4c97jBwUENDAxkbACA6KEoX3R4Grx89NFHGh4eVmVlZcb+yspKHT9+POtjqqqq9Nhjj2nr1q365S9/qbq6Ol1zzTXas2dP1uNbWlpUVlY2ulVXV7v+dwDACMuy1N7eTv6EoYI0uwyTV5QidbFYLOO2bdtj9o2oq6vLWAV1+fLlOnLkiO6//359/etfH3P8pk2btHHjxtHbAwMDBDAAXJdOp9Xc3ByK1XvDzO2ifNSLMZOnPS8VFRWKx+NjellOnDgxpjcml8svv3zcXzklJSUqLS3N2ADAbbkWeIR5Cp1dRr0Ys3kavEydOlVLly7Vjh07Mvbv2LFDK1asyPt5urq6VFVV5XbzACAvTMGNHoJVs3k+bLRx40atW7dOy5Yt0/Lly/XYY4+pt7dXt912m6Qvhn0++OADPf3005KkBx98UAsWLNBFF12koaEhPfPMM9q6dau2bt3qdVMBIKt8puAypBAeI8HqmU4PVnm//eV58HLjjTfq5MmT+pu/+RsdO3ZMF198sdra2jR//nxJ0rFjxzJqvgwNDemuu+7SBx98oOnTp+uiiy7SSy+9pKamJq+bCgBZMQU3WghWzcfyAACQB8rOR4dlWRkTR7LdT/DiPuOWBwCAoGMKbnRQL8Z89LwAgAMs8BgNfX19Wrt2LVPji8jJ9ZvgBQCAcRCsFo+T63dRitQBQDYUAIPpampq+GwaiJwXAEVHATAAhSB4AVB0FAADUAiCFwBFRbVaAIUieAFQVPkUAAOAXAheABQV1WoBFIrgBUBRUQAMQKEIXgAUHdVqARSCOi8Aiq68vFypVCqyBcCobwMUhuAFgG+iVgAsnU6rubmZkvNAgRg2AoAiob4N4A6CFwAoAurbAO4heAGAIqC+DeAeghcAKALq2wDuIXgBgCKgvg3gHoIXACgS6tsgSCzLUnt7u5H5WEyVBoAiiXp9GwRDEKb0x2zbtv1uhJsGBgZUVlam/v5+lZaW+t0cAAACJZlMqrOzM2NmXDweV319vVKplGev6+T6zbARALjM5O52IJegTOkneAEAl6TTaSWTSdXV1ampqUm1tbVKJpPq6+vzu2lAXoIypZ/gBQBcQgVdmMhJT2BQpvQTvACAC4LS3Y7omExPYFCm9BO8AIALgtLdjuiYbE9gEKb0M1UaAFwQlO52RMNIT+CZTu8JHK8XJQhT+ul5AQAXBKW7HdHgRk9gTU2NGhsbjfzsErwAgEuC0N2OaAh7TyDDRgDgkiB0tyMaRnoCxys2F/TPJT0vALKi0Nr4Jjo3Jne3IzrC3BNIzwuADEFY18QvnBsESZh7AlnbCEAGv9Y1CQLODeAdJ9dvghcAoyzLUl1dXc77w/LLzSnODeAtFmYEMCkUWhsf5wYwB8ELgFFhn15ZCM4NYA6CFwCjKLQ2Ps4NYA6CFwAZwjy9slCcG8AMJOwCyCqM0yvdwrkB3MdsI4IXAAAChdlGAAAgtKiwCyAvlmXp0KFDDJUA8B09LwBySqfTSiaTqqurU1NTk2pra5VMJtXX1+d30wBEFMELgJyam5vV2dmZsa+zs1Nr1671qUUAoo5hIwDjsiwrYxHCEcPDw+ro6FBPT49s22Y4CQgpU4eL6XkBMK6JSuLfdNNNDCcBIWT6cDHBC4BxTVQS/4033si4zXASEA6mDxcXJXh5+OGHtXDhQk2bNk1Lly7Vyy+/nPP43bt3a+nSpZo2bZq+/OUv65FHHilGMwGcZqS7+IorrhhTEv+ss7746hgeHs7Yf/pwEoBgGhkuNvn/t+fBy/PPP68NGzbonnvuUVdXl1auXKnGxkb19vZmPf7w4cNqamrSypUr1dXVpbvvvlvf/e53tXXrVq+bCkBju4t//etfa9asWRnHfO1rX8v5HKywDASLZVlqb29XT09PIFZQ97zC7mWXXaZLL71UW7ZsGd23ePFirVmzRi0tLWOO/973vqcXX3xRv/nNb0b33XbbbXrjjTe0d+/eCV+PCrtAYZLJpDo7OzN+dcXjca1YsUKbNm3SokWLZNu26urqxn0Oy7KMSu4DkF06nVZzc3NGYv4VV1yhX//61+M+ZqRXxu0kXmMq7A4NDen1119XQ0NDxv6Ghga98sorWR+zd+/eMccnEgnt379fv/3tb8ccPzg4qIGBgYwNwOTk6i5++eWXR7+sorzC8um/UIGgy5bbsnfvXp177rlZ/3+fe+65SiQSvifxehq8fPTRRxoeHlZlZWXG/srKSh0/fjzrY44fP571+M8++0wfffTRmONbWlpUVlY2ulVXV7v3BwAR46S7OGorLJs++wJwKtePlZMnT2rFihUZ+2fNmjXm8+5XEm9REnZjsVjGbdu2x+yb6Phs+yVp06ZN6u/vH92OHDniQov9w686+Gmi2UWLFi0a/Xd5eblSqZQsy1JbW5ssy1IqlVJ5ebnXzfSF6bMvAKcm+rGyadOm0f/fHR0dOnnypD7//POMY/xK4vU0eKmoqFA8Hh/Ty3LixIkxvSsj5s6dm/X4s88+W+eee+6Y40tKSlRaWpqxBRG/6mCCyQwH1dTUqLGxMfRDRabPvgCcyufHysj/7zM/+2cqdhKvp8HL1KlTtXTpUu3YsSNj/44dO8Z0R41Yvnz5mOO3b9+uZcuWacqUKZ611W/8qoMpojYclI8gzL4AnHLyY8VJr2xR2B577rnn7ClTptiPP/64/c4779gbNmywv/SlL9nvv/++bdu2/f3vf99et27d6PHvvfeePWPGDPvOO++033nnHfvxxx+3p0yZYv/iF7/I6/X6+/ttSXZ/f78nf48Xuru7bUnjbpZl+d1ERJBlWXZbW5tnn7/u7m5Pn99N/B9FWKXTaTuRSGR8nhOJhJ1Op8ccm0gk7Hg8nnFsPB63E4mEK21xcv32PHixbdv+6U9/as+fP9+eOnWqfemll9q7d+8evW/9+vX2lVdemXH8rl277CVLlthTp061FyxYYG/ZsiXv1wpi8NLW1pbzi7Gtrc3vJgKuOXnyZN5flibx+osb8FM+P1acBDqT4eT67Xmdl2ILYp0Xy7KomYHIGK+OTH19vVKplI8ty62vr09r167NqIeRSCTU2toa2iRlIJuenh4dPHjQ1zovBC+GCOoXOuBEGAJ1r764gagzpkgd8keSJKLA9MTXfEoVRGF2FWC6s/1uAL4wUjODX3UIM+NmLPy/bCXSGRICzEXPi2H4VYcwM3VZAUoVAMFC8AKgqEwbIqUAHRA8DBsBKCrThkjzycOhJxQwC8ELAF/U1NQYERSYmocDYHwMGwGINFPzcACMj+DFAVZ8BsLJtDwcALkxbJQHplEC4WZaHg6A3Kiwm4coVb+1LEuHDh3y5cvbz9dGYXjvABSKCrsu8msaZbGHqNLptJLJpOrq6tTU1KTa2lolk0n19fWF+rVRGN47AH4geJlAscuZ+3Ux8LNIFwXCgisI7x25akD4MGw0gWIvJOfHEJWfi+WFYaG+qDL9vSNXDQgWho1cVMxplH4NUfm5WJ7pC/VhfKa/d0HoFQIwOQQveSjWNEq/LgZ+FumiQFhwmfzeUfIfCDeClzyMTKO0LEttbW2yLEupVMr1rme/LgZ+FumiQFhwmfzemd4rBKAwBC8OeL3is58XAz+LdFEgLLhMfe9M7hUCUDgSdg3T19entWvX+pZk6GeRLgqEBZeJ753T5Hdq1QD+cnL9JnhxidtffCZeDIAgyfeHALOSADMQvBQxeOGLDzDbRD8EolRBGzAZwUsRgxe++IDgMr1WDRAl1HkpEqZjAsHGrCQgmAheCmDqFx/l0HE6Pg/jY1YSEEwELwUw7YuPRfJwOj4PEzO5Vg2A8RG8FMC0Lz7KoeN0fB7yY2qtGgDjI2G3QH7XZRlB4iFOx+fBOcoTAP5ycv0+u0htCq2RpQP8/uLLJ/+GL+To4PPgXE1NDecECAiCF5f4/cVnWv4N/MXnAUCYkfPiAhNmc5iWfwN/8XkAEGYELwUwbTYHiYc4HZ8HAGFFwm4BTK2u63f+DczC5wFAELA8QBGCF2ZzhAsrCgOAv1geoAhMra4LZ0wb+gMATIzgZZJMms1hQsJwUFHIDQCCh+BlkkyYzUGvQWHCtLAmASyAKCF4KYDfsznoNShMGIb+CGABRBEJuy7wYzYHCcOFC8M5NHXGGwA4RcJukdXU1KixsdGzC122IYEw9Br4zYShv0KEadgLAJwgeDFYriEBkxKGg8zvob9CEMACiCqCF4PlymkJeq+BKUYW1rQsS21tbbIsS6lUqqgrgk8WASyAqCLnxVD55GNUVFRo7dq16ujoGN2fSCTU2toaiIsvCkfOC4CwIOclBPIZEghyrwHcEeRhLwCYrLP9bgCyczIkUFNTwzBRRI0EsKxfBCBKCF4MNZLTMt6QABconI4AFkCUMGxkMIYEAAAYi54Xl3ixKjFDAgAAjOVpz0tfX5/WrVunsrIylZWVad26dfr4449zPubmm29WLBbL2C6//HIvm1mQYpRn97oIHgAAQeJp8NLc3KwDBw4olUoplUrpwIEDWrdu3YSPSyaTOnbs2OjW1tbmZTMLwvpCAAAUl2fDRr/5zW+USqW0b98+XXbZZZKkf/7nf9by5cvV3d2ds4ZJSUmJ5s6d61XTXDNSnv1Mp5dnp7cEAAB3edbzsnfvXpWVlY0GLpJ0+eWXq6ysTK+88krOx+7atUtz5sxRbW2tbr31Vp04ccKrZhYkrOXZs62lBACAKTzreTl+/LjmzJkzZv+cOXN0/PjxcR/X2NioG264QfPnz9fhw4f113/917r66qv1+uuvq6SkZMzxg4ODGhwcHL09MDDgzh+Qh7CVZ0+n02pubqZiLwDAaI57XjZv3jwmofbMbf/+/ZKkWCw25vG2bWfdP+LGG2/UN77xDV188cVavXq12tvbZVmWXnrppazHt7S0jCYEl5WVqbq62umfNGlhW1+I/B0AQBA47nm54447dNNNN+U8ZsGCBXrzzTf13//932Pu+/DDD1VZWZn361VVVWn+/PnjDmFs2rRJGzduHL09MDBQ1ACmtbV1zPpCQazFQv4OvORFKQEA0eU4eKmoqFBFRcWExy1fvlz9/f169dVX9Yd/+IeSpH/7t39Tf3+/VqxYkffrnTx5UkeOHFFVVVXW+0tKSrIOJxVLWGqx5JO/E8S/C/5iKBKAFzxL2F28eLGSyaRuvfVW7du3T/v27dOtt96qP/qjP8qYaXThhRdq27ZtkqRPPvlEd911l/bu3av3339fu3bt0urVq1VRUaHrr7/eq6a6Iui1WMKWvwMzMBQJwAue1nl59tln9ZWvfEUNDQ1qaGjQV7/6Vf3Lv/xLxjHd3d3q7++X9EWuyFtvvaXrrrtOtbW1Wr9+vWpra7V3717NnDnTy6ZGQq5ZRGHL34H/RoYiT1+bS8ocigSAyfB0eYDZs2frmWeeyXmMbduj/54+fXrWvAsUJt+u+7Dk75gqankfDEUC8AprGxWRXxevXF33qVRqdF9Y8ne8Mtn3z8u8D5MDIoYiAXjGDpn+/n5bkt3f3+93U0adPHnSTiQStqTRLZFI2Ol02vPX7u7uznjdMzfLsjxvQ9AV+v4lEgk7Ho9nPD4ej9uJRMK3NhWLF387gHBycv32NOcFX/AzaTGsVYCLqZD3z6u8j6Akwra2tqq+vj5jH0ORAApF8OIxv5MWC+26j/pSAYW+f14Ej35/ppwYGYq0LEttbW2yLEupVIpp0gAKQvDiMb97PiY7iyidTiuZTKqurk5NTU2qra1VMplUX1+fp+01TaHvnxd5H35/piYj6KUEAJiF4MVjJiQtTqbrPijDEl4r9P1zewp6Op3Wj370o4LaBABBR/DiMRPqpzjtug/SsITX3Hj/3Mz7aG5u1t69e7PeR00eAFFB8FIEpiQt5tt1H8RhCS8V+v65lfcxXlA5YsWKFcYnwkY9hwqAO6jzUgRBq59iwlCXSdx6/2pqagp63ycKKjdt2mRsIixrHAFwU8y2TytxGwIDAwMqKytTf3+/SktL/W5OYCWTSXV2dmb8yo/H46qvr88obHc6kwum+cmt82JZVsa6YNnuN/W8T+bzBCBanFy/GTYKucl20zsZKmFmUnZunxcT8qcmgxwqAG4jeAmpQi+cTvI0mJmUnRfnxZT8KSfIoQLgNoaNQqpY3fRBHsrwktfnJSj5UxKfEQD5Ydgo4orZTR+mX9VuzoTx+rwEqehbUIe7AJiL4CWEihlQhGFmkhc5O2E4L24K4nAXAHMRvIRQMS+ck/lVbVqtDy9yU+htyMQaRwBc5fEK10XnZEntMEskEnY8HrcljW7xeNxOJBKuv1Y6nbYTiUTGayUSCTudTmccd/LkybyOK6bu7u6M9py5WZY16efO97wAAJxdv0nYDam+vj6tXbu2qEXBJkoiNbHWR3t7u5qamsa9v62tTY2NjQW9RpCSayeD+j4A3ODk+k3wEnKmXDhNnXFiaruCgKq5ANzEbCOMMmVWiqmzkshNmTzq+wDwC8ELisLk2TfMhHGOqrkA/ETwgqIwuYeDmTDOmdqTBiAaCF5QNKb3cJgyxBYEJvekAQi/s/1uAKJjpIfDlCRiTN5IT9p4s8d4XwF4iZ4XFB09HOFgek8agPCi5wXApNCTBsAvBC8hRNEwFFNNTQ2fMwBFxbBRiHixwCAAAKYheAkRioYBAKKA4CUkKBoGAIgKgpeQoGgYACAqCF5CgqJhAICoIHgJCZPL7wMA4CaClxChaBgAIAqo8xIiFA0DAEQBwUsIUTQMABBmBC8RRiVeAEAQkfMSQVTiBQAEGcFLBFGJFwAQZAQvEUMlXgBA0BG8RAyVeAEAQUfwEjFU4gUABB3BS8RQiRcAEHQELxFEJV4AQJBR5yWCqMQLAAgygpcIc1qJl6J2AAATMGyECVHUDgBgEk+Dl/vuu08rVqzQjBkzNGvWrLweY9u2Nm/erHnz5mn69Om66qqr9Pbbb3vZTEyAonYAAJN4GrwMDQ3phhtu0O233573Y/7+7/9eDzzwgB566CG99tprmjt3rq699lqdOnXKw5ZiPBS1AwCYxtPg5Yc//KHuvPNOfeUrX8nreNu29eCDD+qee+7Rn/zJn+jiiy/WU089pf/5n//Rz372My+binFQ1A4AYBqjcl4OHz6s48ePq6GhYXRfSUmJrrzySr3yyis+tiy6KGoHADCNUcHL8ePHJUmVlZUZ+ysrK0fvO9Pg4KAGBgYyNriHonYAANM4Dl42b96sWCyWc9u/f39BjYrFYhm3bdses29ES0uLysrKRrfq6uqCXhtjUdQOAGASx3Ve7rjjDt100005j1mwYMGkGjN37lxJX/TAVFVVje4/ceLEmN6YEZs2bdLGjRtHbw8MDBDAuIyidgAAkzgOXioqKlRRUeFFW7Rw4ULNnTtXO3bs0JIlSyR9MWNp9+7d+ru/+7usjykpKVFJSYkn7UEmp0XtkIkifwDgDk9zXnp7e3XgwAH19vZqeHhYBw4c0IEDB/TJJ5+MHnPhhRdq27Ztkr4YLtqwYYN+9KMfadu2bfqP//gP3XzzzZoxY4aam5u9bCoixLIstbe3F22aN0X+AMBdni4PcO+99+qpp54avT3Sm7Jz505dddVVkqTu7m719/ePHvNXf/VX+t///V/9xV/8hfr6+nTZZZdp+/btmjlzppdNRQSk02k1Nzero6NjdF8ikVBra6vKy8s9e91cRf5SqZRnrwsAYRWzbdv2uxFuGhgYUFlZmfr7+1VaWup3c2CQZDKpzs7OjIJ78Xhc9fX1ngURlmWprq4u5/0MIQGAs+u3UVOlAa/4VSmYIn8A4D6CF0SCX0EERf4AwH0EL4gEv4IIivwBgPsIXhAJfgYRFPkDAHeRsBtxUao90tfXp7Vr1xZ9ttEIivwBwPicXL8JXiLKr2nDk+F2gEUQAQDmIXgheJmQH9OGnQpSgAUAKAxTpZHTq6++6su0YadyFXcDAEQXwUsE3X777TnvN6H2iF91WQAA5iN4CZmJ1u2xLEv//u//nvM5TKg9QnE3AMB4CF5CIt/F/yYKCi699FIjklgp7gYAGA/BS0jkmx8yUVDw6KOPut62yaC4GwBgPAQvIeAkP2S8oOCss85SIpHQsmXLitLmfFDcDQCQDcFLCDjND8kWFFx77bXGBQUhm8UPAHDJ2X43AIVzmh9SXl6uVCplfLG2XENhptSiAQAUH0XqQiIIReecsCxLdXV1Oe83MeACAEwOReoiKGz5IUyVBgCMh2GjkAjKUFC+mCoNABgPwUvI1NTUBDpoGTEyK2q8obAw/I0AgMlh2AjGCttQGADAHfS8wFhhGwoDALiD4AXGC8tQGADAHQwbAQCAQCF4AQAAgcKwETAJlmXp0KFD5OEAgA/oeQEcSKfTSiaTqqurU1NTk2pra5VMJtXX1+d30wAgMgheAAdyrbcEACgOghcgT5ZlqaOjI6NoniQNDw+ro6NDO3bs8KllABAtBC9AniZab6mhoYEhJAAoAoIXIE8TrbckMYQEAMVA8ALkaWS9pXg8Pu4xI0NIPT09RWwZAEQLwQvgQLb1lrI5ePBgEVoDANFE8ILQsyxL7e3trvSGjKy31NHRkfO4RYsWFfxaAIDsCF4QWl7WZGloaMg6hBSPx5VIJChcBwAeIniBK9zs3XCL1zVZsg0h1dfXq7W11ZXnBwBkF7Nt2/a7EW4aGBhQWVmZ+vv7VVpa6ndzQi+dTqu5uTljGCWRSKi1tVXl5eW+tcuyLNXV1eW8363ekZ6eHh08eJClAgCgAE6u3/S8oCCmVpydqCaLmwm1NTU1amxsJHABgCIheMGkTVRx1s8hpIlqspBQCwDBRfCCSStm74ZT49VkIaEWAIKP4AWTZnrvBgm1ABBOZ/vdAATXSO9GZ2dnxtBRPB5XfX29770bIzVZSKgFgHCh5wUFCULvBgm1ABAu9LygIPRuAACKjeAFrqipqSFoAQAUBcNGAAAgUAheAABAoBC8AACAQCF4AQAAgeJp8HLfffdpxYoVmjFjhmbNmpXXY26++WbFYrGM7fLLL/eymQAAIEA8DV6GhoZ0ww036Pbbb3f0uGQyqWPHjo1ubW1tHrUQAAAEjadTpX/4wx9Kkp588klHjyspKdHcuXM9aBEAAAg6I3Nedu3apTlz5qi2tla33nqrTpw4Me6xg4ODGhgYyNgAAEB4GRe8NDY26tlnn9W//uu/6ic/+Ylee+01XX311RocHMx6fEtLi8rKyka36urqIrcYAAAUk+PgZfPmzWMSas/c9u/fP+kG3XjjjfrGN76hiy++WKtXr1Z7e7ssy9JLL72U9fhNmzapv79/dDty5MikXxsAAJjPcc7LHXfcoZtuuinnMQsWLJhse8aoqqrS/Pnz1dPTk/X+kpISlZSUjN62bVuSGD4CACBARq7bI9fxXBwHLxUVFaqoqHDeqkk6efKkjhw5oqqqqryOP3XqlCQxfAQAQACdOnVKZWVlOY/xdLZRb2+v0um0ent7NTw8rAMHDkiSFi1apHPOOUeSdOGFF6qlpUXXX3+9PvnkE23evFnf/OY3VVVVpffff1933323KioqdP311+f1mvPmzdORI0c0c+ZMxWIxV/6OgYEBVVdX68iRIyotLXXlOcOM8+Uc58wZzpcznC/nOGfOuHG+bNvWqVOnNG/evAmP9TR4uffee/XUU0+N3l6yZIkkaefOnbrqqqskSd3d3erv75ckxeNxvfXWW3r66af18ccfq6qqSqtWrdLzzz+vmTNn5vWaZ511ls477zx3/5D/V1payofYAc6Xc5wzZzhfznC+nOOcOVPo+Zqox2WEp8HLk08+OWGNl9PHtqZPn66Ojg4vmwQAAALOuKnSAAAAuRC85KGkpEQ/+MEPMmY1YXycL+c4Z85wvpzhfDnHOXOm2OcrZuczJwkAAMAQ9LwAAIBAIXgBAACBQvACAAACheAFAAAECsGLQ3/8x3+s888/X9OmTVNVVZXWrVuno0eP+t0sY73//vu65ZZbtHDhQk2fPl0XXHCBfvCDH2hoaMjvphnrvvvu04oVKzRjxgzNmjXL7+YY5+GHH9bChQs1bdo0LV26VC+//LLfTTLanj17tHr1as2bN0+xWEwvvPCC300yVktLi/7gD/5AM2fO1Jw5c7RmzRp1d3f73SyjbdmyRV/96ldHi9MtX75c7e3tnr8uwYtDq1at0s9//nN1d3dr69atOnTokP70T//U72YZ691339Xnn3+uRx99VG+//bb+4R/+QY888ojuvvtuv5tmrKGhId1www26/fbb/W6KcZ5//nlt2LBB99xzj7q6urRy5Uo1Njaqt7fX76YZ69NPP9Ull1yihx56yO+mGG/37t369re/rX379mnHjh367LPP1NDQoE8//dTvphnrvPPO09/+7d9q//792r9/v66++mpdd911evvttz19XaZKF+jFF1/UmjVrNDg4qClTpvjdnED48Y9/rC1btui9997zuylGe/LJJ7VhwwZ9/PHHfjfFGJdddpkuvfRSbdmyZXTf4sWLtWbNGrW0tPjYsmCIxWLatm2b1qxZ43dTAuHDDz/UnDlztHv3bn3961/3uzmBMXv2bP34xz/WLbfc4tlr0PNSgHQ6rWeffVYrVqwgcHGgv79fs2fP9rsZCJihoSG9/vrramhoyNjf0NCgV155xadWIcxG1t3j+yo/w8PDeu655/Tpp59q+fLlnr4WwcskfO9739OXvvQlnXvuuert7dWvfvUrv5sUGIcOHdI//dM/6bbbbvO7KQiYjz76SMPDw6qsrMzYX1lZqePHj/vUKoSVbdvauHGjrrjiCl188cV+N8dob731ls455xyVlJTotttu07Zt2/T7v//7nr4mwYukzZs3KxaL5dz2798/evxf/uVfqqurS9u3b1c8Htef/dmfKWqjb07PmSQdPXpUyWRSN9xwg771rW/51HJ/TOZ8IbtYLJZx27btMfuAQt1xxx1688031dra6ndTjFdXV6cDBw5o3759uv3227V+/Xq98847nr6mp6tKB8Udd9yhm266KecxCxYsGP13RUWFKioqVFtbq8WLF6u6ulr79u3zvJvMJE7P2dGjR7Vq1SotX75cjz32mMetM4/T84WxKioqFI/Hx/SynDhxYkxvDFCI73znO3rxxRe1Z88enXfeeX43x3hTp07VokWLJEnLli3Ta6+9pn/8x3/Uo48+6tlrErzod8HIZIz0uAwODrrZJOM5OWcffPCBVq1apaVLl+qJJ57QWWdFr8OvkM8YvjB16lQtXbpUO3bs0PXXXz+6f8eOHbruuut8bBnCwrZtfec739G2bdu0a9cuLVy40O8mBZJt255fEwleHHj11Vf16quv6oorrlB5ebnee+893Xvvvbrgggsi1evixNGjR3XVVVfp/PPP1/33368PP/xw9L65c+f62DJz9fb2Kp1Oq7e3V8PDwzpw4IAkadGiRTrnnHP8bZzPNm7cqHXr1mnZsmWjvXi9vb3kUOXwySef6ODBg6O3Dx8+rAMHDmj27Nk6//zzfWyZeb797W/rZz/7mX71q19p5syZo718ZWVlmj59us+tM9Pdd9+txsZGVVdX69SpU3ruuee0a9cupVIpb1/YRt7efPNNe9WqVfbs2bPtkpISe8GCBfZtt91m/9d//ZffTTPWE088YUvKuiG79evXZz1fO3fu9LtpRvjpT39qz58/3546dap96aWX2rt37/a7SUbbuXNn1s/T+vXr/W6accb7rnriiSf8bpqx/vzP/3z0/+Pv/d7v2ddcc429fft2z1+XOi8AACBQopd8AAAAAo3gBQAABArBCwAACBSCFwAAECgELwAAIFAIXgAAQKAQvAAAgEAheAEAAIFC8AIAAAKF4AUAAAQKwQsAAAgUghcAABAo/wcOEWU7/Djj9AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 创建一个非线性数据集\n",
    "rnd = np.random.RandomState(0) #设置随机数种子\n",
    "X = rnd.uniform(-3, 3, size=100) #random.uniform，从输入的任意两个整数中取出size个随机数\n",
    "\n",
    "#生成y的思路：先使用NumPy中的函数生成一个sin函数图像，然后再人为添加噪音\n",
    "y = np.sin(X) + rnd.normal(size=len(X)) / 3 #random.normal，生成size个服从正态分布的随机数\n",
    "\n",
    "#使用散点图观察建立的数据集是什么样子\n",
    "plt.scatter(X, y,marker='o',c='k',s=20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:37:28.444305Z",
     "start_time": "2024-03-06T03:37:28.434700Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100,)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:37:30.167757Z",
     "start_time": "2024-03-06T03:37:30.158856Z"
    }
   },
   "outputs": [],
   "source": [
    "X = X.reshape(-1, 1) #sklearn只接受二维以上数组作为特征矩阵的输入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:37:31.358420Z",
     "start_time": "2024-03-06T03:37:31.347871Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 1)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:37:41.842302Z",
     "start_time": "2024-03-06T03:37:41.830755Z"
    }
   },
   "outputs": [],
   "source": [
    "#使用原始数据进行建模\n",
    "LinearR = LinearRegression().fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA580lEQVR4nO3deXhV5bX48e9KCEIQLQZtQcyJA1iHgsrQ4IStghanahWLUdHeNoJ6W0OxDulF9Gdsq70gOBZFsCV1FuutVBG11oFBRgfACZIQsRWUUWRIsn5/7H3iSThTzrT3OVmf59lPzrCHd58ke539rncQVcUYY4yJJM/rAhhjjPE3CxTGGGOiskBhjDEmKgsUxhhjorJAYYwxJioLFMYYY6KyQGF8TUT+KSI/T9f6rbb9toj8S0S2isj/JrKPZIiIishh7uMHROR/Ml2GVuW5SUQeSsN+PT830zYdvC6AyR4iUgN8G2gEtgEvANeo6rYMHf9y4OeqemKaDlEObAD2UY87GKnq6EweT0ROAWaqaq+QMtyegv1eTqvfWabPzSTP7ihMW52tqnsDxwDHAjd6W5yUCgArEgkSIuLrL11+L5/xNwsUJiGq+m/gRZyAAYCIlIrIWyKySUSWu99Sg+9dLiKr3WqdNSJS5r4+QURmhqxX4lbBtLiwicgRwAPAYBHZJiKbohTvUBFZKCKbReRvIrJfrDKKyAxgFPAbd/+nicheInKXiKxzl7tEZC93/VNEpF5ErheRfwPTRSRPRG4QkU9E5AsReSL02K2JyHUi8pm775+1em+GiNzmPu4uIn93y/yliLwuInnueweJyDMist495j0hn/ebIjJJRL4EJrjn80cRqROR/7hVQJ1FpAvwD6Cne+7bRKRn6O9GRO4JeW+biDSIyAT3veA5bxWRFSJyXrTfWei5uc9/ISIfu+f2nIj0DHlPRWS0iHwkIhtF5F4RkSi/e5MGFihMQkSkF/Aj4GP3+YHA88BtwH7AOOBpEdnfvRBNAX6kql2B44FlbTmeqq4ERgPzVHVvVf1WlNUvA34G9AQa3GNHLaOqXg5UA3e4+58LVAKlOMGwHzAI+G3Icb7j7ieAU231S+DHwBD32BuBe8MVUETOcI8/FOgNnBblfH4N1AP741T93QSoiOQDfwdqgRLgQOCxkO2+D6wGDgCqgD8AfdzzOcxdf7yqfoXzu1znnvveqroutACqek3wPeBE99z+5r79CXASsC9wCzBTRHrE8zsTkR8CvwNGAD3cc3ms1WpnAQNxfgcjgNOjfFYmDSxQmLZ6VkS2AmuBz4Gb3dcvAWar6mxVbVLVl4BFwHD3/SbgaBHprKqfqer7aSzjX1T1PfcC+D/ACPeiGquMrZUBt6rq56q6HucieGnI+03Azaq6U1W/Bq4EKlW1XlV3AhOACyJU+4wApoeUc0KU89mNcxENqOpuVX3drR4bhBOQrlPVr1R1h6q+EbLdOlW9W1UbgB3AL4AKVf1SVbcCtwM/jXLcPYjI/sCzwH+r6lIAVX1SVde5n+njwEdu2eJRBjysqkvcz+xGnDuQkpB1fq+qm1S1DniVkLtYkxkWKExb/di9KzgF+C7Q3X09AFzoVo9scqsZTgR6uBfCi3C+XX4mIs+LyHfTWMa1IY9rgQK3nBHLGGE/Pd3tQ/fVM+T5elXdEfI8AMwK2fdKnMT/tyPsu3U5I7kT585tjjjVdze4rx8E1LqBIJzQ/e8PFAKLQ8r3gvt6XESkAHgK+KuqPhby+mUisixkv0fzzd9FLC0+Y7dhxBc4dztB/w55vB3YO94ym9SwQGESoqqvATOAP7ovrcX5Jv+tkKWLqv7eXf9FVR2Kc1FeBTzobvcVzgUs6DvRDhtn8Q4KeVyM8418Q6wyhrEO5+Ifuq/QKpnW5VmLU70Wuv9OqvppmH1/FqacYanqVlX9taoeApwNjBWRU93jFUe4Y2ldvg3A18BRIWXb161KCncu4dwNbCWk+k1EAji/y2uAIrd66T0gmEeItd8Wn7FbTVkEhPvMjEcsUJhk3AUMFZFjgJnA2SJyuojki0gnN+HbS5z+Cee4F4GdOE1rG919LANOFpFiEdmX6K2o/gP0EpGOMcp1iYgcKSKFwK3AU6raGK2MEfbzKPBbN8/SHRjv7iOSB4Aq9+KJu925EdZ9Arg8pJw3R1gPETlLRA5zk7hbcD67RmAhTsD5vYh0cc/nhHD7UNUmnAv6JBE5wN3vgSISrO//D1Dk/g7CleFKnNzLxe6+grrgBIP17npX4NxRBMX6nf0VuEJEjhGnocDtwAJVrYn0eZjMs0BhEubW2/8Z+B9VXQuci5NoXY/zbfc6nL+xPJyE7DrgS5wLzlXuPl4CHgfeARbjJGcjeQV4H/i3iGyIst5fcO52/g10wkkyE6OM4dyGk8N4B3gXWOK+Fslk4DmcKqKtwHychPIeVPUfOIH2FZxqpVei7Lc3MBcnwM4D7lPVf7rB72ycxHQdTsL7oij7ud491nwR2eLu83C3PKtwAuNqtwqpZ6ttRwKHAOtCWj7dpKorgP91y/Uf4HvAmyHbRf2dqerLOHmkp3GC3qG0MW9i0k9s4iJjjDHR2B2FMcaYqCxQGGOMicoChTHGmKgsUBhjjIkqJwcK6969u5aUlHhdDGOMyRqLFy/eoKphO2DmZKAoKSlh0aJFXhfDGGOyhohEHB3Aqp6MMcZEZYHCGGNMVBYojDHGRJWTOYpwdu/eTX19PTt27Ii9sskanTp1olevXhQUFHhdFGNyVrsJFPX19XTt2pWSkhJsgqzcoKp88cUX1NfXc/DBB3tdHGNyVrupetqxYwdFRUUWJHKIiFBUVGR3iSYh1dXVlJSUkJeXR0lJCdXV1V4Xybc8DRQi8rCIfC4i70V4/xRx5j1e5i7jkzxeMpsbH7LfqUlEdXU15eXl1NbWoqrU1tZSXl5uwSICr+8oZgBnxFjndVU9xl1uzUCZjDE5rrKyku3bt7d4bfv27VRWVnpUIn/zNFCo6r9w5idoF/be25lMbN26dVxwwQUel8Z7DzzwAH/+85+9LoZph+rq6tr0enuXDcnswSKyHGfSm3Gq+n64lUSkHCgHKC6OOKukL/Ts2ZOnnnoqrcdoaGigQ4fwv95o78VDVVFV8vKS+54xevTopLY3JlHFxcXU1u7ZEdnv1w6veF31FMsSIKCq/XDm63020oqqOlVVB6jqgP33j3u+eE/U1NRw9NHObJEzZszg/PPP54wzzqB379785je/aV5vzpw5DB48mOOOO44LL7yQbdu2AXDrrbcycOBAjj76aMrLywlOPnXKKadw0003MWTIECZPntzimBMmTKC8vJxhw4Zx2WWXsX79en7yk58wcOBABg4cyJtvOpOSrV+/nqFDh3Lcccdx5ZVXEggE2LBhAzU1NRxxxBFcddVVHHfccaxdu5Y777yTgQMH0rdvX26+2ZnJ86uvvuLMM8+kX79+HH300Tz++OMA3HDDDRx55JH07duXcePGNZfpj390ptxetmwZpaWl9O3bl/POO4+NGzc2n9P111/PoEGD6NOnD6+//npafiemfamqqqKwsLDFa4WFhVRVVXlUIn/z9R2Fqm4JeTxbRO4Tke6qGm0azJjklvQkQPXmxGYLXLZsGUuXLmWvvfbi8MMP57//+7/p3Lkzt912G3PnzqVLly784Q9/YOLEiYwfP55rrrmG8eOdvP6ll17K3//+d84++2wANm3axGuvvRb2OIsXL+aNN96gc+fOXHzxxVRUVHDiiSdSV1fH6aefzsqVK7nlllv44Q9/yI033sgLL7zA1KlTm7f/4IMPmD59Ovfddx9z5szho48+YuHChagq55xzDv/6179Yv349PXv25Pnnnwdg8+bNfPnll8yaNYtVq1YhImzatGmPsl122WXcfffdDBkyhPHjx3PLLbdw1113Ac4d0MKFC5k9eza33HILc+fOTehzNiaorKwMcHIVdXV1FBcXU1VV1fy6acnXgUJEvgP8R1VVRAbh3AF94XGxUu7UU09l332dOe2PPPJIamtr2bRpEytWrOCEE04AYNeuXQwePBiAV199lTvuuIPt27fz5ZdfctRRRzUHiosuijxl8jnnnEPnzp0BmDt3LitWrGh+b8uWLWzdupU33niDWbNmAXDGGWfQrVu35nUCgQClpaWAc7czZ84cjj32WAC2bdvGRx99xEknncS4ceO4/vrrOeusszjppJNoaGigU6dO/PznP+fMM8/krLPOalGuzZs3s2nTJoYMGQLAqFGjuPDCC5vfP//88wHo378/NTU1bflojYmorKzMAkOcPA0UIvIocArQXUTqgZuBAgBVfQC4ABgjIg3A18BPNQWTfCf6zT9d9tprr+bH+fn5NDQ0oKoMHTqURx99tMW6O3bs4KqrrmLRokUcdNBBTJgwoUU/gi5dukQ8Tuh7TU1NzJs3rzlwBEX7eEO3V1VuvPFGrrzyyj3WW7x4MbNnz+bGG29k2LBhjB8/noULF/Lyyy/z2GOPcc899/DKK69EPE5rwc8n+NmY9KqurqayspLa2lry8/NpbGwkEAjYN+52zOtWTyNVtYeqFqhqL1WdpqoPuEECVb1HVY9S1X6qWqqqb3lZ3kwqLS3lzTff5OOPPwacpnsffvhhc1Do3r0727ZtSzgpPmzYMO65557m58uWLQPgxBNP5IknngCcu4ZgrqC1008/nYcffrg5b/Lpp5/y+eefs27dOgoLC7nkkksYN24cS5YsYdu2bWzevJnhw4dz1113NR8raN9996Vbt27N+Ye//OUvzXcXJrNC+xcANDY2Alg/g3bO11VP7dn+++/PjBkzGDlyJDt37gTgtttuo0+fPvziF7/ge9/7HiUlJQwcODCh/U+ZMoWrr76avn370tDQwMknn8wDDzzAzTffzMiRI3n88ccZMmQIPXr0oGvXrs0BIWjYsGGsXLmyuTps7733ZubMmXz88cdcd9115OXlUVBQwP3338/WrVs599xz2bFjB6rKpEmT9ijPI488wujRo9m+fTuHHHII06dPT+i8THLC9S8ICvYzsLuK9kdSUJPjOwMGDNDWExetXLmSI444wqMSZY+dO3eSn59Phw4dmDdvHmPGjNnjDsBv7HebOnl5eVGrH0WEpqamDJbIZIqILFbVAeHeszsK00JdXR0jRoygqamJjh078uCDD3pdJJNBkfoXhL5v2h+/96MwGda7d2+WLl3K8uXLefvttxOu2jLZKVz/giA/9jOwgf0yo10FilysZmvv7HeaWmVlZUydOpVAIAA4Lc3AaRo9depU3+Qnqqur6d69O5dccokN7JcB7SZHsWbNGrp27WpDjeeQ4HwUW7dutfko2pFgy6xISfdAIGD9bRJgOQqgV69e1NfXs379eq+LYlIoOMOdaT+itcwCG9gvHdpNoCgoKLBvncZ4LNiZL5lhM2IFAku4p167ylEYY7yTqsmCogUCPybcc4EFCmNMRqRqsqBILbOKiop8lXDPJRYojDEZkarJgkJbZokIgUCAmTNnsmHDBgsSadJuWj0ZY7xVUlIStjOftVLyh2itnuyOwhiTtHg6vtlkQdnLAoUxJinxJqnDVRlZTiE7WNWTMSYpVqWUG6zqyRiTNqlKUhv/skBhjElKpH4N1vEtd1igMMYkxZLUuc8ChTEmKZakzn2WzDbGGGPJbGOMaQ+27NySlv1aoDDGmCymqrxe+zrnP34+h0w+hG27tqX8GBYojMlS7X0a0PZ+/rsbd/Pou48y6KFBnDzjZGatmsXWXVtZUL8g9QdT1Zxb+vfvr8bkspkzZ2phYaECzUthYaHOnDnT66KpqlO+QCCgIqKBQCDl5fL7+afTxq836h1v3KG9JvZSJqBMQLvf0V3HvzJeP9v6WcL7BRZphGuqJbONyUJ+7g0dbqrSwsLClLaE8vP5p8snX37C5AWTeXjpw3y1+ysAjuh+BBWlFVzS9xI6F3ROav/RktmeBgoReRg4C/hcVY8O874Ak4HhwHbgclVdEmu/FihMrsvLyyPc/66I0NTU5EGJvpGJi7ifzz+VVJU3177JxHkTeXbVsyjOOQ89ZCgVpRWcftjp5ElqMgh+bvU0Azgjyvs/Anq7SzlwfwbKZIzvZbI3dFtzAZkY0iPXe4MH8w/ff+j7nDT9JGatmkVBfgFXHHMF74x+hzmXzuFHvX+UsiARU6Q6qUwtQAnwXoT3/gSMDHn+AdAj1j4tR2FyXabq6BM5TiAQaLF+cAkEAp6WKxukK/8QD6LkKPweKP4OnBjy/GVgQIR1y4FFwKLi4uJUf4bG+E66E8aqkS/6+fn5EY+b6EW8reeTifPPlI+/+Fh/OfuX2qWqS3OA+O4939Wpi6bq9l3bM1KGbA4Uz4cJFP1j7dPuKIwJr60XVxEJGyiiBYGZM2dqUVFR8/tFRUVxXfRz8Q4hmqamJn299nU977HzVCZIc4A47c+n6ewPZ2tjU2NGy5PNgcKqnoxJkVRWI0WqVkr0gp+J6qpkpeoOZlfDLv3rO3/VgVMHNgeHjv+vo17x7BW6/N/LU1zq+GVzoDgT+AcgQCmwMJ59WqAwZk/RqpEiXfTCXfjDLSIS9RixLviR7lyC+/VaKu54wuUfiv5QpP/zyv+kPf8QD98GCuBR4DNgN1AP/BcwGhjtvi/AvcAnwLuR8hOtFwsUxuwpWjVStIte6Dfp/Pz8qIEg0Qu+3+8okilfKvMP6czL+DZQpGuxQGHMnmJVI8Vz0Yv1zTrRC6rfcxRtDYCR8g9HjT5KD+h5QEIX+nR/RhYojDExq5GifesP/SZbVFSkRUVFYS92yVzM/NyKKd4AGC3/cPu9tyd1oU/3XZcFCmOMqjoX41jVR+G2acsFzqsLfjqPG+sziCf/kOyFPt15HAsUxuSwRPoftOXC7/f8gWpmqq7Cfc6ffPlJ3PmHZC/0dkdhgcKYhGSic5vfWySpZjaYBfMP5z9+fpv6PyRbRstRWKAwJiGZuEAm0js70zIRzHY17NJH3310j/zD5c9eHlf/h1Rc6K3VkwUKY9osExfIePpSeN1CKZ0BM5h/OGjiQUn3f/Bzwt4ChTE+lszFI1NVLm3pS+GFdFTLRMo//GnRnzI2/lImWaAwxqeSvcB50f/ArzmLtgTcSOsmmn/IBRYojPGpVNwRZLo6I5ky+6HqJVJwvfr3Vyecf8gFFiiM8alM5RhSeXFOpqWVH3pfR+yhvm9y+YdsZ4HCGJ9Kd44hXRfnRIKPX/pjRBvzKlfzD/GIFig8nTM7XWzObJMtqqurKS8vZ/v27c2vFRYWMnXqVMrKypLefybmr46X1/NcqypvrX2Lof2H8vWGr/d4v7i4OOxn1V74ec5sY9q1srIypk6dSiAQQEQIBAIpCxKQmfmr4+XVPNe7G3fz2HuPUTqtlBOnn8jXJ38NBS3XKSws5Pbbb09rObKZBQpjPFZWVkZNTQ1NTU3U1NSkLEiAdxfncKqqqigsLGzxWmFhIVVVVWk53qYdm/jjW3/k0CmHMvLpkSz8dCFFnYv47dW/5Z7770lbcM5JkeqkggtwQjyv+WmxHIUxDr8kkEPLk+5WT8H+D3vfvvce/R++2vVVyo+XLD+0BFNNMpkNLInnNT8tFiiM+YZfLkTp1NTUpG/UvqHnP36+5t2S1xwgTn3kVH3+w+fT2v8hkUEZg+sXFRVpQUGBLwJ5QoECGAz8GlgLjA1ZJgDLI23nh8UChTHtQ3D8pUEPDmoODgW3Fujlz16uyz5btsf6XjcVjndqWS96uScaKIYAN+NMVXpzyDIW6B1pOz8sFiiMSR8/3KFs/Hqj3vnmnXuMv/Tbl3+r67asC7tNOqrh2trkN2IfDh/0co8WKGI2jxWRgKpmVZsxax5rTHqkuzlvLKs3rmby/Mk8vOxhtu3aBsB3u3+XitIKLul7CYUFhRG3TUdT4bY2+Y20firLlKhkm8fOEJFXWi8pLqMxJoOqq6spKSkhLy+PkpISqqur49qusrKyRZAA2L59O5WVlekoJuDUerxZ9yY/eeIn9L67N1MWTmHbrm2cevCpPH/x87x/1fuU9y+PGiQgPU2F29qqLJ7WZulsCZawSLcawQXoH7KcAEwE7oi1nZeLVT2ZbJSpKp1kqmAyOSDg7sbd+ti7j8Wdf4glHT3DU5Gj6NixY8Q5yDOJVA/hAbyWyHaZWixQmGyTyWasyVwwMzEMRzD/UDypOO78Qzz8MpyJH3I84SQVKID9QpbuwOnAB7G283KxQGGyTSbHQUrmriCdAe2TLz/RX/3jVy36Pxx+9+H6wNsPpKz/g18v0n6QbKBYA6x2f34EzAFOjLWdl4sFCpNtMlmlk2xQSuXFNh39HywYJCblVU9+XyxQmGyTyTsKP/TWjpR/GDVrVEL5hyA/nFu2SvaOohNO34lngKeBCqBTrO3iWYAzgA+Aj4Ebwrx/CrAZWOYu4+PZrwUKk20yfYHz6lt3uvIPQX4ZyjwbRQsU8fSjeALYCsx0XxoJdFPVC6NuGIOI5AMfAkOBeuBtYKSqrghZ5xRgnKqe1ZZ9Wz8Kk42qq6uprKykrq6O4uJiqqqqcmagutUbVzNlwRSmLZ3W3P/h8KLDqSit4NJ+l8Zs2hovr4cyz2bR+lF0iGP7w1W1X8jzV0VkeQrKNQj4WFVXu4V8DDgXWBF1K2NyVFlZWZsCg98Di6oz/8PE+RN5dtWzNKlzoT714FMZO3gsZxx2BnmS2gGsI80p4cVoubkknt/SUhEpDT4Rke8Db6bg2AfijCMVVO++1tpgEVkuIv8QkaMi7UxEykVkkYgsWr9+fQqKZ4x/BXtI19bWoqrU1tZSXl4ed8e5dGpoauDx9x5vnv/hmZXPkC/5jOo3imVXLmPuZXMZ3nt4yoMEJDaUeaKdD9uVSHVS+k2eYCXQBNS4SxPwPvAu8E6s7aPs90LgoZDnlwJ3t1pnH2Bv9/Fw4KN49m05CpPr/FgXHyn/UPlyZUryD/FqS/7Fkt/fINmxnmIEmoTGgRKRwcAEVT3dfX6ju7/fRdmmBhigqhui7dtyFCbX+akufs3GNUxeMDnt+Yd08NNUsV5LNkdxm6pe2mqHf2n9WgLeBnqLyMHAp8BPgYtbHec7wH9UVUVkEE5V2RdJHteYrOd1XbyqMq9+HhPnTWTWqlnN+YcfHvxDxpaO5Ue9f5SWqqVU89NUsX4WT6BokRcQkQ444z4lRVUbROQa4EUgH3hYVd8XkdHu+w8AFwBjRKQB+Br4qca6BTKmHaiqqgo7imu6B5NraGrg6RVPM3H+RBZ+uhCAgrwCLu17KRWlFfT7Tr8Ye/AXrwNu1ohUJwXciNMstgHY4j7eivON/neRtvPDYjkKk0le9UnI5HE3fb1J//jmHz3PP6Sa5Si+QZId7nwdFMItFihMpuT6hWb1l6vTPv6S12zID0e0QBFPMvvkCHci/0r+fiY9LJltMiUXk6GaI/kH0zbJJrOvC3ncCaej3GLghykomzFZLZeSobmWfzCpEzNQqOrZoc9F5CDgjrSVyJgskgvJ0M07NvPQkoeYsnAKdZudALdf5/0YM2AMVw+8mh5de3hcQuO1RO4f64GjU10QY7JRIj2BvRCu9/GajWu49oVr6TWpF+NeGkfd5joOLzqc+8+8n7UVa7nth7dZkDBAHHcUInI3TpIOnMByDJCKsZ6MyXrBsZX8POZScLiPYFPa2tpaRv1sFI1nNUJfZx3LP5ho4klmjwp52gDUqGoqxnpKG0tmG/ONSAl39oXLZlxGRWkFx3znmIyXy/hLUslsVX1ERDoCfdyXPkhl4Ywx6bN5x2Zq68KPsiNbhEd+/EiGS2SyUcx7THdOiI+Ae4H7gA8jNZk1xvjDmo1rqHihgl6TejlDa4aRTQl34+0ot/E0j/1fYJiqfgAgIn2AR0nBMB7GmNSJ1P/hyJ8eySePfMLOHTub1/Vjwt1EFi7PVF5eDpCZfFiknnjBhTBDiYd7zU+L9cw27cnuxt36+HuP6/cf/L5yPsq+Tg/xLvt30ap7q1TVeh9nu0wMK0+SPbMfdgv1F/elMqCDql6RjsCVCpbMNu3BHv0f3gH+D9j9zTqFhYVMnTrVV62wzJ5izVaYiWHloyWz4wkUewFXAycCAvwLuE9Vd0bd0EMWKEwuW7NxDVMWTOGhpQ81z//Qp6gPG6o28OW/v9xj/WweTqQ9aF2tBHsG+EwMFZNUoMhGFihMrtEo4y9VlFYwvPdwOuR3SPu3TpN68QSBeIJJspId68kY45GGpgaeWfkME+dNZMGnCwBn/KVL+l6yR/+HXBhOpD2KZ7wwrzt2WhdM06552eQwms07NjNx3kQOnXIoFz11EQs+XcB+nffjphNvoubaGh758SMc851jWpR/27ZtdOzYscV+rHWT/0UK5K1fLysro6amhqamJmpqajKbd4qU5c7mxVo9mXj4cS6J1V+u1mv/ca12vb1r8/wPfe7uo/e/ff8e8z+EK39BQYEWFRVZ66Ys4pe/Q5Js9dQHZ6jxACFVVarq22HGLUdh4uGnuSTmrZ3HxPkTeWblM2HzD+HGX/JT+U1yYrV6yoRkWz0tBx7AmYOiMfi6qi5OZSFTyQKFiUcmmhxGEyn/MPJ7I+Maf8nr8pvckmwyu0FV709xmYzxnFfJ3807NjNt6TQmL5jcYv6H0f1Hc/Wgq+nZtWdc+7HktcmUeJLZ/yciV4lIDxHZL7ikvWTGpFmm55IIjr900KSD+PWcX1O3uY4+RX24b/h91F1bR9WpVXEHCcieuTBM9ovnjiI4zHjolKgKHJL64hiTOZlqchgu//CDkh8wdvDYiPmHeHjdZNK0H9bhzhhXKhOKyeYfjMm0aDmKeIYZLxCRX4rIU+5yjYgUpL6Yxngn2PO1trYWVaW2tpYrrriC7t27t6mPRbz9H4xpKy/7/MTT6ukhoAAIznByKdCoqj9Pc9kSZncUpq0izgIXItqQCcHxl6YtncbWXVsBZ/yla79/LZf1u4wuHbukpdymffB6CI+4mseqar9YryVYsDOAyUA+8JCq/r7V++K+PxzYDlyuqkti7dcChYlXsLopVpAIat1HIV35B2NCeT0oYDx/xY0icmjIzg4hpD9FEoXKx5k170fAkcBIETmy1Wo/Anq7SzlgzXRNyoRWN8Wrrq6OhqYGnnz/SQZPG8zxDx/PUyueIl/yuazfZSy9cimvjHqFs/qcZUHCJCW0qinS32htbW1mqqIiddkOLsCpQB3wT+A1oAb4Qazt4tjvYODFkOc3Aje2WudPwMiQ5x8APWLt24bwMPGINBlMtKXbt7tpYFKgeXiNbr/vpjfNvUk/3fJpRstuExHltnDDesRakh32gyhDeMR7Ud8L6Av0A/aKZ5s49nkBTnVT8PmlwD2t1vk7cGLI85eBARH2Vw4sAhYVFxcn/GEly/6Bs4eIRPynKyoq0o4dO7Z8vQBnBrkJaO8pvfW+hffptp3bMl5uv4wNZNInkS8xJDnjXUKBAvih+/P8cEuk7eJdgAvDBIq7W63zfJhA0T/Wvr26o7B/4OwSa3rJmyffrIXd3d/nvk6Q+MGMH+hzq57TxqZG35bbZL9oX2JivZeoaIEiWiXqEPfn2WGWs6JsF6964KCQ572AdQms4xuVlZUtWiUAbN++ncrKSo9KZKKJ1LP5nDHnMHjaYG7ZeAvbr9lOh1s7cOn0S1lyzxJeGfUKZx9+dnP+wYsmi/HMX2CyW6RhWAKBAE1NTQQCgTZtl7RIESTdC06v8NXAwUBHYDlwVKt1zgT+gTMFaymwMJ59e3VHESnSJxPlTXqFVhV2+3Y3LSorijv/4NUdpN1R5L5Yf1vp+NsjmRwF8CtgH/di/RCwBBgWa7t4Fpxmrx8CnwCV7mujgdHuY8FpGfUJ8C4R8hOtF68Chf0DZ581G9doxQsVLeZ/iDf/kKnfd+u815gxY6yKsx2Ile9MdT402UCx3P15OvAcTkJ7SaztvFwsR2FieavuLb3giQs075a85gDR1vxDJu4gI/1NjRkzxhpNmJSKFijiGRRQ3J/DgemqutztCGdasUHa/K2hqYFZK2cxcf5E5tfPB6BDXgfKvldGRWkFx/Y4tk37y8Qw35HyXrNnz7bJiUzGxNMzezpwIE4uoR9OL+p/qmr/9BcvMdYz24QKzv8wZcEUajc7F/ZunboxZsCYNs3/0FomhlWwyYlMpiTbM/u/gBuAgaq6HWfcpytSWD7f8HLQLZN6NZtqGPvi2Ob5H2o319J7v97cN/w+1lasbfP8D62VlZUxdepUAoEAIkIgEEhpkIDIdyc2OZHJqEh1UsEFOAHo4j6+BJgIBGJt5+WSSI7Cz/mFdHTiy+WOgW/VvaUXPnFhi/zDKTNO8bz/QyL8/HdpcgtJJrPfwclT9HMf/wp4LdZ2Xi6JBIpUtmBJ5UU4HReKXLz47G7crU+894SWPlTaHBw63NpBL33mUl2ybklC+8xUMM106xZjwkk2UCxxf44H/iv0Nb8uiQSKVLVgSfVFOB1NMHOpGe/mHZt14lsT9xh/6ca5N2r95vqE95upYJqLQdtkp2iBIp5k9mvAC8DPgJOA9cAyVf1ezHotjySSzE7VML6pHg44HcnMXEiQ1myqYcqCKTy05KHm+R9679ebitKKlMz/kIlhnTN5HGNiSTaZfRGwE/iZqv4bpwXUnSksny+kaqL6VA+vkI5kZjYnSOetnceIJ0dw6JRDmTR/Elt3beWUklN47qfPseqaVYwZOCYlkwRlapgMG47DZIVItxqhCxAATnMfFwJd49nOqyXRDnepqAtOdbWO5SjSk3+IJVPVc7GOY/kJkykkmaP4BfA28In7vDfwcqztvFxS2TO7rf+o6bqwt8dWT+nKP8TDDzmKbAvoJrslGyiW4QzatzTktXdjbeflkqpAkeg/ajZchP0smfGXUsnrVk+51OjA+F+ygWKB+3Op+7MD8E6s7bxcUhUo0v2PagGlpXlr56W0/0O2f742GrHJpGiBIp6xnl4TkZuAziIyFLgK+L84tst66Uw0th7+oba2lvLycoB2NTZUcPylSfMnMa9+HuCMv3Tx9y6morSC43ocl9B+c+HzzcRYUsbEI57msQL8HBiG0/HuRZyZ6aJv6KFUjfWUzqaL7b1Z5JadW5i2ZBqTF0xuMf7S6AGjuXrg1Ry4z4FJ7T8XPt9MjCVlTFC05rGxqp3ygPeirePHJZU5ioKCgha3/QUFBSmpwmiv1QprNq7RsS+MbZF/OGzKYXrvwntTmn/Ilc8326vPTPYg0aonVW0SkeUiUqyq7bJhd+sR1VM1wnp7q1aYXz+fifMm8vTKp2lSp1PfKSWnMLZ0LGf2ObN5atFUyZXPt6yszO4ejOfi+e/sAbwvIi+LyHPBJd0F84PKykp27drV4rVdu3alZA7sVHXw84twI+82NDXw5PtPcvy04xk8bTBPrniSPMnjkr6XsLh8Ma+OerXF/NOplGufrzGeinSrEVyAIeGWWNt5uaSq6ikd1RehVQlFRUVaVFSU9dUK4ZoRF+xVsMf805no/9C6XFZtY0x8SKZ5bDYufm0em6sdqCJ9TuybnvxDOBYUjElOUoEC2ApsabWsBWYBh8Ta3ovF6w53keRqB6pod16ZmP8hVwOwMZkULVDEUzk8EbgOZzDAXsA44EHgMeDhOLbPWsnOYNa63j5cchWycwC4hqYGnlrxFMdPOx7dJ3xL6eLi4rTkH1qLNK90KnJJxpj4+lEsUNXvt3ptvqqWishyVe2X1hImwA9zZodrAy8ihPu8s6ltf7D/w5SFU6jZVANA4cpCdj27i4adDc3rZbK9fy4Mm26M15IdZrxJREaISJ67jAh5z7ed7rwW7luuqu7RvDZbWuLUbqrl1y/+ml4TezF2zlhqNtVw2H6Hce/we/n8z58zY9qMtM4dHU02D5tuTFaIVCcVXIBDcIbs2IAzadH/AYcBnYETY23vxZLK0WMTFaneHjcnkS1J13lr5+mIJ0e0GH9pyPQh+rdVf/PN/NOWozAmeSQz1pOqrgbOjvD2G0nEqJwWqcNXNlQzNTQ18OyqZ5k4b2JKx19Kl+CdS2VlJXV1dRQXF1NVVWUd1YxJkZhVTyLSx+1s9577vK+I/DaZg4rIfiLykoh85P7sFmG9GhF5V0SWiYgnSYdwHcnikY0dvrbs3MKkeZPofXdvLnzyQubVz6Nbp27ccMIN1Pyqhr+c9xffBYmgsrIyampqaGpqoqamxoKEMakU6VYjuACvAYNoOR9FUuM/AXcAN7iPbwD+EGG9GqB7W/fvl+ax2dK2v2ZjTUbGXzLG+BdJ9qN42/25NOS1ZbG2i7HPD4Ae7uMewAcR1vM0UPip30M6gk425B+MMZkRLVDEMx/FBhE51L1IIiIXAJ8lcPMS6tuq+hmAqn4mIgdEWE+BOW5i+E+qOjXSDkWkHCiH1LV28cvE96mcWyFS/mHk0SOpKK2gf8/+qS28MSb7RYog+s23+kOAucB24FOcBHYgju3mAu+FWc4FNrVad2OEffR0fx4ALAdOjnVczcE7ilSUIzj/dMldJc13D9/6/bf0hpduyOj4S7kgW6oUjWkLUjHWE9AF6IozFWpZvNtF2FdcVU+ttpkAjItn/37JUSRyvHAXoGQGJ4yUf7hnwT26defWtJxHLrOmuCZXJRQogH2AG4F7gKE4s9tdg5M3+Fuk7eJZgDtpmcy+I8w6XYCuIY/fAs6IZ/+p7EcR6eKd6m+V0S5AidxRzF87X0c8OULzb8nfI//Q0NiQVFnbel7xfk7Z8E3dL3eZxqRaooHib8AM4ErgCeAlnBZQx0TaJt4FKAJeBj5yf+7nvt4TmO0+PsStbloOvA9Uxrv/dHe4S8e3ymgXoHiPt7txtz75/pN6/LTjm4NDh1s7aNnTZbro00XJnnabteVzSuYzzWSAyZWZ84xpLdFA8W7I43xgY/Abvt+XdAeKdHyrjHUBCr2zyM/PbxFENu/YrJPmTfJd/qEtn1Oin2mmq4LsjsLkqkQDxZJoz/28pDtQpONbZTwXoHAXxQ57ddBOIzr5Mv/Qls8pkc905syZzUEzUxduy1GYXJVooGjkm/kntgINIY+3RNrOD0s23lHEcwGKNkGQF/mHWNJ5RxHu88pUVVA25FKMaauEAkU2L9mYowjuN9IFaHfj7qyrH09njiJi0MxQVZAFC5NrLFCkQSovFNH21SL/sG/0i6IfL17xlmnmzJlaVFTUfE5FRUVRyx9tdN50VwVZ9ZPJRRYofCzSReeuqXfp2BfG6j6/26c5/3DApQdoQaeCsBeobL54JVL2SHcU+fn5aT9nS2ibXGSBwsei5R2CAeLk6Sfrsyuf1YbGhojf0P1w8Ur0jiaRsnsZGLOtCtCYeFig8LFoVSht6f8Qb/PadFVLJXPhTvTC61VVmx+CsjGpZoHCpzbv2KzdvtMt7EXnwIMObNO+UtFhLxnJXDz9euGN1is/W6v5jInEAoXP1Gys0V+/+Gsn/3A+SkHyydhUDwHSVslUx/jxwhurTH5sOGBMMixQ+ES48ZdOnn6yVtxRocXFxUlfdNIxqGC8kg1Gfrvw+vUux5h0iRYoxHk/twwYMEAXLfJk5tQ9NDY1OvM/zJ/IW2vfApz5Hy466qKMzf9QUlKS9vm7W8+ZAc7Ur1OnTs3KaUnz8vII978hIjQ1NXlQImPSS0QWq+qAcO/FM3GRScCWnVt4eOnDTFkwhTWb1gDwrU7f4sr+V3LNoGvotU+vjJWlqqoq7EU8lfN3B4NBZWUldXV1FBcXU1VVlZVBApzJr8IF11RNimVMVol0q5HNi5dVTy3yD2710qGTD9W7F9zt6fhLfqva8Ts/5k2MSScsR5F+kfIPwf4P4djF29/s92Pak2iBwnIUSYiUfxhx1AgqSisY0DNsdR+Qe3X6xpjsFi1HYYEiAanIP2QiwWyMMfGyZHaK1G2uY8qCKTy45EG27NwCwKHdDuXa0mu5/JjL2bvj3vHvq66uTa8bY4xXLFDEYeGnC5k4byJPrXiKRm0E4OTAyYwtHctZfc4iPy+/zfu0VjXGmGxhgSKCSPmHi4++OGb+IR6ZaLJqjDGpkOd1Afxm686tTJ4/md539+aCJy/grbVv8a1O3+I3x/+G1b9cTfX51UkHCXD6HUydOpVAIICIEAgEWiSyq6urKSkpIS8vj5KSEqqrq5M+pjHGJMKS2a76LfXcNf+ulOQfkmUtoowxmWatnuLw0icvMWzmMMDJP1SUVnB2n7MTyj8ky1pEGWMyzVo9xeG0Q07juuOvY8RRI1JStZQMaxFljPETCxQuEeGOoXd4XQzAWkQZY/zFktk+VFVVRWFhYYvXQltEWaLbGJNJngQKEblQRN4XkSYRiVjPIyJniMgHIvKxiNyQyTJ6KVqLqGCiu7a2FlWltraW8vJyCxbGmLTxJJktIkcATcCfgHGqukfmWUTygQ+BoUA98DYwUlVXxNq/n+ajSDVLdBtj0sF3yWxVXQlOXiCKQcDHqrraXfcx4FwgZqDIZZboNsZkmp9zFAcCa0Oe17uvtWuREtqW6DbGpEvaAoWIzBWR98Is58a7izCvRawnE5FyEVkkIovWr1+fWKGzQKxEdzazJL0x/pS2QKGqp6nq0WGWv8W5i3rgoJDnvYB1UY43VVUHqOqA/fffP5mi+1qsoT8yKZUXdkvSG+NfnvbMFpF/EjmZ3QEnmX0q8ClOMvtiVX0/1n5zOZntF6keZsSS9MZ4K1oy26vmseeJSD0wGHheRF50X+8pIrMBVLUBuAZ4EVgJPBFPkDCZUVlZ2SJIAGzfvp3KysqE9mdJemP8y8Z6MgnJy8sj3N+OiNDU1NTm/dkdhTHe8t0dhYkt04ndth4v1a2vcjlJb0zWU9WcW/r376/ZbObMmdqxY0fFaeWlgHbs2FFnzpyZtuMVFha2OF5hYWHU4yWyTTzlCAQCKiIaCATSdr7GmD0BizTCNdWqnnyoe/fufPHFF3u8XlRUxIYNG1J+vESrfaqrq6msrKSuro7i4mKqqqpsvgxjspRVPflQtKqecEEi2uvJSjSRXFZWRk1NDU1NTdTU1FiQMCZHWaDwgN/6DFhvb2NMNBYoPBCraWlRUVHY7SK9nqzhw4fvMe6WJZKNMUEWKDwQq6pn8uTJFBQUtHivoKCAyZMnp7ws1dXVPPLIIy2auooIo0aNsqokYwxggcITsap6ysrKmD59eothOqZPn56WC3e4uxtVZfbs2Sk/ljEmO1mrJw+keviLZKS645wxJjtZqyef8dPAfpbINsbEYoHCI35pWmo9oo0xsVigaOf8dHdjjPEnCxQmI3c3NimRMdnLkzmzTfvSOnkf7GAI2J2LMVnA7ihM2kXqYDhq1Ci7szAmC1igMGkXqYNhY2OjTXdqTBawQGHaJJFcQ7SmtsnMimeMyQwLFCZuiQ5mGK4Jbiib7tQYf7NAkSMy0aoo0Xmyg01w8/Pzw75vnfuM8Tdr9ZQDMtWqKNF5K0LLEW7oEuvcZ4y/2R1FDkj0m35bJTvch3XuMyY72aCAOSBTA/v5aTBDY0xq2aCAOS5TA/vZHYEx7ZMFihyQyYH9/DKYoTEmcyxQ5AD7pm+MSSfLURhjjPFfjkJELhSR90WkSUTCFsxdr0ZE3hWRZSJiV35jjPGAV/0o3gPOB/4Ux7o/UNUNaS6PMcaYCDwJFKq6Epzmm8YYY/zN78lsBeaIyGIRKY+2ooiUi8giEVm0fv36DBXPGGNyX9ruKERkLvCdMG9Vqurf4tzNCaq6TkQOAF4SkVWq+q9wK6rqVGAqOMnshAptjDFmD2kLFKp6Wgr2sc79+bmIzAIGAWEDRajFixdvEJHaBA/bHciVnEiunEuunAfYufhVrpxLMucRiPSGbwcFFJEuQJ6qbnUfDwNujWdbVd0/ieMuitRELNvkyrnkynmAnYtf5cq5pOs8vGoee56I1AODgedF5EX39Z4iMttd7dvAGyKyHFgIPK+qL3hRXmOMac+8avU0C5gV5vV1wHD38WqgX4aLZowxphW/t3rywlSvC5BCuXIuuXIeYOfiV7lyLmk5j5wcwsMYY0zq2B2FMcaYqCxQGGOMicoCRRgi8v9E5B13MMI5ItLT6zIlQkTuFJFV7rnMEpFveV2mRMU7kKSficgZIvKBiHwsIjd4XZ5EicjDIvK5iLzndVmSISIHicirIrLS/dv6lddlSpSIdBKRhSKy3D2XW1K6f8tR7ElE9lHVLe7jXwJHqupoj4vVZiIyDHhFVRtE5A8Aqnq9x8VKiIgcATThDCQ5TlWzajRhEckHPgSGAvXA28BIVV3hacESICInA9uAP6vq0V6XJ1Ei0gPooapLRKQrsBj4cZb+TgTooqrbRKQAeAP4larOT8X+7Y4ijGCQcHXBGXMq66jqHFVtcJ/OB3p5WZ5kqOpKVf3A63IkYRDwsaquVtVdwGPAuR6XKSHuMDpfel2OZKnqZ6q6xH28FVgJHOhtqRKjjm3u0wJ3Sdl1ywJFBCJSJSJrgTJgvNflSYGfAf/wuhDt2IHA2pDn9WTpRSkXiUgJcCywwOOiJExE8kVkGfA58JKqpuxc2m2gEJG5IvJemOVcAFWtVNWDgGrgGm9LG1ms83DXqQQacM7Ft+I5lywWbkz9rLxTzTUisjfwNHBtq9qErKKqjap6DE7NwSARSVm1oG/Hekq3Ngxa+FfgeeDmNBYnYbHOQ0RGAWcBp6rPE1KpGEjSx+qBg0Ke9wLWeVQW43Lr858GqlX1Ga/LkwqquklE/gmcgTNJXNLa7R1FNCLSO+TpOcAqr8qSDBE5A7geOEdVt3tdnnbubaC3iBwsIh2BnwLPeVymds1NAE8DVqrqRK/LkwwR2T/YqlFEOgOnkcLrlrV6CkNEngYOx2llUwuMVtVPvS1V24nIx8BewBfuS/OzsfUWOANJAncD+wObgGWqerqnhWojERkO3AXkAw+rapW3JUqMiDwKnIIzpPV/gJtVdZqnhUqAiJwIvA68i/O/DnCTqs6OvJU/iUhf4BGcv6084AlVjWu07bj2b4HCGGNMNFb1ZIwxJioLFMYYY6KyQGGMMSYqCxTGGGOiskBhjDEmKgsUpl0RkW2x12rzPktE5OIo79/pjuh5ZwL7PsZtVmuMZyxQGJO8EiBioACuBI5T1esS2PcxuPPIx0sc9r9tUsb+mEy7JCKniMg/ReQpd86OarenLiJSIyJ/cMf3Xygih7mvzxCRC0L2Ebw7+T1wkjt/SUWr4zyHMwLxAhG5yO1B+7SIvO0uJ7jrDRKRt0RkqfvzcLcH963ARe6+LxKRCSIyLmT/77l3NCXuvAr3AUuAg0TkOvcY76R6fgLTvligMO3ZscC1wJHAIcAJIe9tUdVBwD04vamjuQF4XVWPUdVJoW+o6jnA1+57jwOTgUmqOhD4CfCQu+oq4GRVPRZntOLb3eHIxwOPh2wfzeE4c0Qc6z7ujTO8+TFAf3ceCWParN0OCmgMsFBV6wHc4ZlLcCZ8AXg05OekPbZM3GnAke7NC8A+7qQ5+wKPuOOMKc58Am1VGzJRzTB3Weo+3xsncPwr0YKb9ssChWnPdoY8bqTl/4OGedyAexfuVlN1TOCYecBgVf069EURuRt4VVXPc+dG+GeE7ZvL4OoU8vir0F0Cv1PVPyVQRmNasKonY8K7KOTnPPdxDdDffXwu33zr3wp0jXO/cwiZ30REjnEf7gsEB568PGT91vuuAY5ztz0OODjCcV4EfubOtYCIHCgiB8RZRmNasEBhTHh7icgC4FdAMEH9IDBERBYC3+ebb/DvAA3iTGxfseeuWvglMMBNMK8AgqP53gH8TkTexBkBNOhVnKqqZSJyEc7cCfu5VWVjcObh3oOqzsGZS2WeiLwLPEX8wcyYFmz0WGNaEZEaYICqbvC6LMb4gd1RGGOMicruKIwxxkRldxTGGGOiskBhjDEmKgsUxhhjorJAYYwxJioLFMYYY6L6/7E2vSIbY0Z3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#放置画布\n",
    "fig, ax1 = plt.subplots(1)\n",
    "\n",
    "#创建测试数据：一系列分布在横坐标上的点\n",
    "line = np.linspace(-3, 3, 1000, endpoint=False).reshape(-1, 1)\n",
    "\n",
    "#将测试数据带入predict接口，获得模型的拟合效果并进行绘制\n",
    "ax1.plot(line, LinearR.predict(line), linewidth=2, color='green',\n",
    "         label=\"linear regression\")\n",
    "\n",
    "#将原数据上的拟合绘制在图像上\n",
    "ax1.plot(X[:, 0], y, 'o', c='k')\n",
    "\n",
    "#其他图形选项\n",
    "ax1.legend(loc=\"best\")\n",
    "ax1.set_ylabel(\"Regression output\")\n",
    "ax1.set_xlabel(\"Input feature\")\n",
    "ax1.set_title(\"Result before discretization\")\n",
    "# plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从图像上可以看出，线性回归无法拟合出这条带噪音的正弦曲线的真实面貌，只能够模拟出大概的趋势。可见，使用线性回归模型来拟合非线性数据的效果并不好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 如何判断数据是线性数据还是非线性数据呢？\n",
    ">> 对于回归问题，数据若能分布为一条直线，则是线性的，否则是非线性<br>\n",
    ">>对于分类问题，数据分布若能使用一条直线来划分类别，则是线性可分的，否则数据则是线性不可分的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型在线性和非线性数据集上的表现为我们选择模型提供了一个思路：当我们获取数据时，我们往往希望使用线性模型来对数据进行最初的拟合（线性回归用于回归，逻辑回归用于分类），如果线性模型表现良好，则说明数据本身很可能是线性的或者线性可分的，如果线性模型表现糟糕，那毫无疑问我们会投入非线性模型的怀抱，就不必浪费时间在线性模型上了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不过这并不代表着我们就完全不能使用线性模型来处理非线性数据了。在现实中，线性模型有着不可替代的优势：计算速度异常快速，所以也还是存在着我们无论如何也希望使用线性回归的情况。因此，我们有多种手段来处理线性回归无法拟合非线性问题的问题，接下来我们就来看一看。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**多项式回归就是一种专门用于解决”线性回归只能处理线性数据“问题的改进版线性回归。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "多项式变化是一种通过增加自变量上的次数，而将特征矩阵的维度增加的方法。只要我们设定一个自变量上的次数（大于1），就可以实现一个非常抽象的目的：将原本非线性的数据投入到高维空间当中，并且让数据在这个高维空间当中变得线性。这种方法可以非常容易地通过sklearn中的类PolynomialFeatures来实现。我们先来简单看看这个类是如何使用的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*class* `sklearn.preprocessing.PolynomialFeatures`(*degree=2*, *interaction_only=False*, *include_bias=True*,*order='C'*)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 参数             | 含义                                      |\n",
    "| ---------------- | ----------------------------------------- |\n",
    "| degree           | 多项式中的次数，默认为2                   |\n",
    "| interaction_only | 布尔值是否只产生交互项，默认为False       |\n",
    "| include_bias     | 布尔值，是否产出与截距项相乘的$x_0$，默认 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:41:27.215904Z",
     "start_time": "2024-03-06T03:41:27.207962Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:41:30.415408Z",
     "start_time": "2024-03-06T03:41:30.403889Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [2],\n",
       "       [3]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#如果原始数据是一维的\n",
    "X = np.arange(1,4).reshape(-1,1)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "#二次多项式，参数degree控制多项式的次方\n",
    "poly = PolynomialFeatures(degree=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1.],\n",
       "       [1., 2., 4.],\n",
       "       [1., 3., 9.]])"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#接口transform直接调用\n",
    "X_ = poly.fit_transform(X)\n",
    "X_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 3)"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  1.,  1.,  1.],\n",
       "       [ 1.,  2.,  4.,  8.],\n",
       "       [ 1.,  3.,  9., 27.]])"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#三次多项式\n",
    "PolynomialFeatures(degree=3).fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不难注意到，多项式变化后数据看起来不太一样了：首先，数据的特征（维度）增加了，这正符合我们希望的将数据转换到高维空间的愿望。其次，维度的增加是有一定的规律的。不难发现，如果我们本来的特征矩阵中只有一个特征$x$，而转换后我们得到：\n",
    "\n",
    "![10](https://pictes.oss-cn-beijing.aliyuncs.com/%E5%BE%AE%E8%AF%BE%20-%20sklearn/week%209%20Linear%20Model/10%20%E4%B8%80%E7%BB%B4%E5%A4%9A%E9%A1%B9%E5%BC%8F.PNG)\n",
    "\n",
    "这个规律在转换为二次多项式的时候同样适用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原本，我们的模型应该是形似$y=ax+b$的结构，而转换后我们的特征变化导致了模型的变化。线性回归是会对每个特征拟合出权重$w$的，所以当我们拟合高维数据的时候，我们会得到下面的模型："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$y = w_0x_0 + w_1x + w_2x^2 + w_3x^3，(x_0=1)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由此推断，假设多项式转化的次数是n，则数据会被转化成形如：$[1,x,x^2,x^3,...,x^n]$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "而拟合出的方程也可以被改写成："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$y = w_0x_0 + w_1x + w_2x^2+w_3x^3...w_nx^n，(x_0 = 1)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个就是“**多项式回归**”的表达式，个过程看起来非常简单，只不过是将原始的$x$上的次方增加，并且为这些次方项都加上权重$w$，然后增加一列所有次方为0的列作为截距乘数的$x_0$，参数include_bias就是用来控制$x_0$的生成的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  1.,  1.],\n",
       "       [ 2.,  4.,  8.],\n",
       "       [ 3.,  9., 27.]])"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#三次多项式，不带与截距项相乘的x0\n",
    "PolynomialFeatures(degree=3,include_bias=False).fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为什么我们会希望不生成与截距相乘的x0呢？对于多项式回归来说，我们已经为线性回归准备好了x0，但是线性回归并不知道，所以它并不会把多项式生成的x0当作是截距项。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_t = PolynomialFeatures(degree=3).fit_transform(X)\n",
    "rnd = np.random.RandomState(0) #设置随机数种子\n",
    "y = rnd.randn(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  1.,  1.,  1.],\n",
       "       [ 1.,  2.,  4.,  8.],\n",
       "       [ 1.,  3.,  9., 27.]])"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 5.27355937e-15, -5.96011571e-01, -1.03690610e+00,  3.34690676e-01])"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成了多少个系数？\n",
    "LinearRegression().fit(X_t,y).coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.062279340310302"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看截距\n",
    "LinearRegression().fit(X_t,y).intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性回归并没有把多项式生成的x0当作是截距项,所以我们可以选择：\n",
    "- 关闭多项式回归中的include_bias\n",
    "- 或者关闭线性回归中的fit_intercept"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2.49402132,  0.4457948 , -1.60516412,  0.42940035])"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LinearRegression(fit_intercept=False).fit(X_t,y).coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LinearRegression(fit_intercept=False).fit(X_t,y).intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不过，这只是一维状况的表达，大多数时候我们的原始特征矩阵不可能会是一维的，至少也是二维以上，很多时候还可能存在上千个特征或者维度。现在我们来看看原始特征矩阵是二维的状况："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [2, 3],\n",
       "       [4, 5]])"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.arange(6).reshape(3, 2)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.,  0.,  1.],\n",
       "       [ 1.,  2.,  3.,  4.,  6.,  9.],\n",
       "       [ 1.,  4.,  5., 16., 20., 25.]])"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#尝试二次多项式\n",
    "PolynomialFeatures(degree=2).fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们仔细看，是可以看出这样的规律的：\n",
    "\n",
    "![11 二维多项式二次](https://pictes.oss-cn-beijing.aliyuncs.com/%E5%BE%AE%E8%AF%BE%20-%20sklearn/week%209%20Linear%20Model/11%20%E4%BA%8C%E7%BB%B4%E5%A4%9A%E9%A1%B9%E5%BC%8F%E4%BA%8C%E6%AC%A12.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1.,   0.,   1.,   0.,   0.,   1.,   0.,   0.,   0.,   1.],\n",
       "       [  1.,   2.,   3.,   4.,   6.,   9.,   8.,  12.,  18.,  27.],\n",
       "       [  1.,   4.,   5.,  16.,  20.,  25.,  64.,  80., 100., 125.]])"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#尝试三次多项式\n",
    "PolynomialFeatures(degree=3).fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这次生成的数据有这样的规律：\n",
    "\n",
    "![12 二位多项式三次](https://pictes.oss-cn-beijing.aliyuncs.com/%E5%BE%AE%E8%AF%BE%20-%20sklearn/week%209%20Linear%20Model/12%20%E4%BA%8C%E7%BB%B4%E5%A4%9A%E9%A1%B9%E5%BC%8F%E4%B8%89%E6%AC%A1.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不难发现：**当我们进行多项式转换的时候，多项式会产出到最高次数为止的所有低高次项**。比如如果我们规定多项式的次数为2，多项式就会产出所有次数为1和次数为2的项反馈给我们，相应的如果我们规定多项式的次数为n，则多项式会产出所有从次数为1到次数为n的项。注意，$x_1x_2$和$x_1^2$一样都是二次项，一个自变量的平方其实也就相当于是$x_1x_1$，所以在三次多项式中$x_1^2x_2$就是三次项。\n",
    "\n",
    "在多项式回归中，我们可以规定是否产生平方或者立方项，其实如果我们只要求高次项的话，$x_1x_2$会是一个比$x_1^2$更好的高次项，因为$x_1x_2$和$x_1$之间的共线性会比$x_1^2$与$x_1$之间的共线性好那么一点点（只是一点点），而我们多项式转化之后是需要使用线性回归模型来进行拟合的，就算机器学习中不是那么在意数据上的基本假设，但是太过分的共线性还是会影响到模型的拟合。因此sklearn中存在着控制是否要生成平方和立方项的参数interaction_only，默认为False，以减少共线性。来看这个参数是如何工作的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 2)"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [2, 3],\n",
       "       [4, 5]])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.,  0.,  1.],\n",
       "       [ 1.,  2.,  3.,  4.,  6.,  9.],\n",
       "       [ 1.,  4.,  5., 16., 20., 25.]])"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PolynomialFeatures(degree=2).fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  0.,  1.,  0.],\n",
       "       [ 1.,  2.,  3.,  6.],\n",
       "       [ 1.,  4.,  5., 20.]])"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PolynomialFeatures(degree=2,interaction_only=True).fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对比之下，当interaction_only为True的时候，只生成交互项"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从之前的许多次尝试中我们可以看出，随着多项式的次数逐渐变高，特征矩阵会被转化得越来越复杂。不仅是次数，当特征矩阵中的维度数（特征数）增加的时候，多项式同样会变得更加复杂，并且这种维度的增加并不能用太简单的数学公式表达出来。**因此，多项式回归没有固定的模型表达式**，多项式回归的模型最终长什么样子是由数据和最高次数决定的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们就来看看多项式回归的根本作用：处理非线性问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "之前我们说过，是希望通过这种将数据投影到高维的方式来帮助我们解决非线性问题。那我们现在就来看一看多项式转化对模型造成了什么样的影响："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:55:21.353920Z",
     "start_time": "2024-03-06T03:55:21.344235Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures as PF\n",
    "from sklearn.linear_model import LinearRegression\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:55:31.994476Z",
     "start_time": "2024-03-06T03:55:31.983125Z"
    }
   },
   "outputs": [],
   "source": [
    "rnd = np.random.RandomState(0) #设置随机数种子\n",
    "X = rnd.uniform(-3, 3, size=100)\n",
    "y = np.sin(X) + rnd.normal(size=len(X)) / 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:55:33.251929Z",
     "start_time": "2024-03-06T03:55:33.241974Z"
    }
   },
   "outputs": [],
   "source": [
    "#将X升维，准备好放入sklearn中\n",
    "X = X.reshape(-1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:55:36.076122Z",
     "start_time": "2024-03-06T03:55:36.062644Z"
    }
   },
   "outputs": [],
   "source": [
    "#创建测试数据，均匀分布在训练集X的取值范围内的一千个点\n",
    "line = np.linspace(-3, 3, 1000, endpoint=False).reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:55:47.500525Z",
     "start_time": "2024-03-06T03:55:47.482150Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5548358879190791"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#原始特征矩阵的拟合结果\n",
    "LinearR = LinearRegression().fit(X, y)\n",
    "#对训练数据的拟合\n",
    "LinearR.score(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6668186456042835"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#对测试数据的拟合\n",
    "LinearR.score(line,np.sin(line))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:56:40.876526Z",
     "start_time": "2024-03-06T03:56:40.866338Z"
    }
   },
   "outputs": [],
   "source": [
    "#多项式拟合，设定高次项\n",
    "d=5\n",
    "#进行高此项转换\n",
    "poly = PF(degree=d)\n",
    "X_ = poly.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:56:48.999803Z",
     "start_time": "2024-03-06T03:56:48.989928Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 6)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:57:24.051856Z",
     "start_time": "2024-03-06T03:57:24.041787Z"
    }
   },
   "outputs": [],
   "source": [
    "line_ = PF(degree=d).fit_transform(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:57:30.171328Z",
     "start_time": "2024-03-06T03:57:30.161603Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 6)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "line_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:57:45.511958Z",
     "start_time": "2024-03-06T03:57:45.493528Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8168228629635071"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#训练数据的拟合\n",
    "LinearR_ = LinearRegression().fit(X_, y)\n",
    "LinearR_.score(X_,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:58:00.533896Z",
     "start_time": "2024-03-06T03:58:00.526472Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9870970106481279"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#测试数据的拟合\n",
    "LinearR_.score(line_,np.sin(line))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们将这个过程可视化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T03:58:08.640177Z",
     "start_time": "2024-03-06T03:58:08.518407Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHWCAYAAAD6oMSKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAACx4UlEQVR4nOzdd1hT5xcH8O8lMkVFZYgmgtS96t6IuLUqitSBe7a1VnAPnC2Ioyq4Z50FB/CzjtYdFPeoW+sERcStgIOR8P7+uCYYSSCBTDif5+GRO3LvyUAO7zgvxxhjIIQQQgghJs/M0AEQQgghhBDtoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEEIIIaSAoMSOEA1s2rQJHMfh4sWLKs+Ji4sDx3HYtGmT/gLToujoaHAcJ/8SCARwcHBA165dc3zeBY3svY6LizN0KFqj7LM5e/ZscBxnuKAKmYL4uSLGpYihAyCkoHF2dsaZM2fwzTffGDqUfJk7dy48PT2RkZGBy5cvY86cOfDw8MCVK1dQqVIlQ4enc9999x3OnDkDZ2dnQ4eiU8OHD0fHjh0NHQYhREsosSNEyywtLdGkSRNDh5Gjjx8/wsbGJsdzKlWqJH8e7u7usLOzw6BBg7Bt2zbMmTNHH2HKqROvtjk4OMDBwUGv99QGTV8roVAIoVCow4iUk0qlkEgksLS01Pu9CSnIqCuWEC3Lqbvr5s2b6Nu3L0qUKAEnJycMHToUSUlJCo9njGHlypWoU6cOrK2tUbJkSfj4+ODhw4cK5x0+fBheXl4QCoWwsrJCxYoV8cMPP+DVq1cK58nu/e+//8LHxwclS5bMU2tigwYNAADPnz9X2H/v3j34+vrC0dERlpaWqFatGlasWJHt8Tdv3kT79u1hY2MDBwcH/Pzzz9i/fz84jkN0dLT8vFatWqFmzZo4ceIEmjVrBhsbGwwdOhQAkJycjAkTJqBChQqwsLBAuXLl4O/vjw8fPijca9euXWjcuDFKlCgBGxsbuLm5ya8BAJmZmQgMDESVKlVgbW0NOzs71K5dG6GhofJzVHWZ/fHHH/j2229hZWWFUqVKoUePHrh9+7bCOYMHD4atrS3u37+Pzp07w9bWFiKRCOPHj0daWlqur3VmZiYWLFiAqlWrwtLSEo6Ojhg4cCCePHmicF5Or9XTp0/Rq1cvFCtWDCVKlEDv3r3x7NmzbPdS1hXr6uqKLl264MCBA6hXrx6sra1RtWpV/PHHHwrnvXz5EqNGjUL16tVha2sLR0dHtG7dGjExMQrnyX4mFixYgMDAQFSoUAGWlpY4fPgw7Ozs8MMPP2SLKy4uDgKBAAsXLlT6GmVkZMDR0REDBgzIduzdu3ewtrbGuHHj5K9nbu+3MrJhCdu2bcO4ceNQpkwZWFtbw8PDA5cvX852/p49e9C0aVPY2NigWLFiaNeuHc6cOZPjPX777TcUKVIE8fHx2Y4NHToUpUuXRmpqao7XIORLlNgRokc9e/ZE5cqVERkZiSlTpiAsLAxjx45VOOeHH36Av78/2rZti927d2PlypW4efMmmjVrppBUPXjwAE2bNsWqVatw6NAhzJw5E+fOnUOLFi2QkZGR7d7e3t6oWLEidu3ahdWrV2sce2xsLACgcuXK8n23bt1Cw4YNcePGDSxatAj79u3Dd999hzFjxii06iUmJsLDwwN37tzBqlWrsGXLFqSkpGD06NFK75WYmIj+/fvD19cXf//9N0aNGoWPHz/Cw8MDmzdvxpgxY/DPP/9g8uTJ2LRpE7p16wbGGADgzJkz6N27N9zc3LB9+3bs378fM2fOhEQikV9/wYIFmD17Nvr27Yv9+/djx44dGDZsGN69e5fjaxAcHIxhw4ahRo0aiIqKQmhoKK5du4amTZvi3r17CudmZGSgW7duaNOmDf766y8MHToUS5Yswfz583N9rX/66SdMnjwZ7dq1w549e/Dbb7/hwIEDaNasWbbEXdlr9enTJ7Rt2xaHDh1CcHAwdu3ahTJlyqB379653lvm6tWrGD9+PMaOHYu//voLtWvXxrBhw3DixAn5OW/evAEAzJo1C/v378fGjRvh5uaGVq1aKSTrMkuXLsWxY8fw+++/459//kHNmjUxdOhQ/Pnnn9n+wFm5ciUsLCwUEvIvmZubo3///oiMjERycrLCsfDwcKSmpmLIkCEA8v5+y0ybNg0PHz7E+vXrsX79ejx9+hStWrVS+GMrLCwMXl5eKF68OMLDw7Fhwwa8ffsWrVq1wsmTJ1Ve+4cffkCRIkWwZs0ahf1v3rzB9u3bMWzYMFhZWakVJyEAAEYIUdvGjRsZAHbhwgWV58TGxjIAbOPGjfJ9s2bNYgDYggULFM4dNWoUs7KyYpmZmYwxxs6cOcMAsEWLFimcFx8fz6ytrdmkSZOU3jMzM5NlZGSwR48eMQDsr7/+ynbvmTNnqvUcxWIxA8B27NjBMjIy2MePH9mpU6dYlSpVWPXq1dnbt2/l53bo0IEJhUKWlJSkcI3Ro0czKysr9ubNG8YYYxMnTmQcx7GbN28qnNehQwcGgInFYvk+Dw8PBoAdPXpU4dzg4GBmZmaW7bWPiIhgANjff//NGGPs999/ZwDYu3fvVD7HLl26sDp16uT4Osje69jYWMYYY2/fvmXW1tasc+fOCuc9fvyYWVpaMl9fX/m+QYMGMQBs586dCud27tyZValSJcf73r59mwFgo0aNUth/7tw5BoBNmzZNvk/Va7Vq1apsnwPGGBsxYoTKz+aXXFxcmJWVFXv06JF836dPn1ipUqXYDz/8oDJ2iUTCMjIyWJs2bViPHj3k+2U/E9988w1LT09XeMyDBw+YmZkZW7JkicK9SpcuzYYMGaLyXowxdu3aNQaArV27VmF/o0aNWP369eXb6rzfysh+FurVqyf/GWWMsbi4OGZubs6GDx/OGGNMKpWysmXLslq1ajGpVCo/LyUlhTk6OrJmzZrJ9339uWKM/7w4OjqytLQ0+b758+czMzMzhfMIUQe12BGiR926dVPYrl27NlJTU/HixQsAwL59+8BxHPr37w+JRCL/KlOmDL799luFVpAXL17gxx9/hEgkQpEiRWBubg4XFxcAyNY1CPCthZro3bs3zM3NYWNjg+bNmyM5ORn79++HnZ0dACA1NRVHjx5Fjx49YGNjoxBv586dkZqairNnzwIAjh8/jpo1a6J69eoK9+jbt6/Se5csWRKtW7dW2Ldv3z7UrFkTderUUbhXhw4dFLpzGzZsCADo1asXdu7ciYSEhGzXb9SoEa5evYpRo0bh4MGD2Vp8lDlz5gw+ffqEwYMHK+wXiURo3bo1jh49qrCf4zh07dpVYV/t2rXx6NGjHO8jFosBINt9GjVqhGrVqmW7j7LXSiwWo1ixYtk+b76+vjne+0t16tRB+fLl5dtWVlaoXLlytvhXr16NevXqwcrKSv45PHr0qNLPYLdu3WBubq6wz83NDV26dMHKlSvlra5hYWF4/fq1yhZdmVq1aqF+/frYuHGjfN/t27dx/vx5hZa+vLzfX/L19VXornZxcUGzZs3k79WdO3fw9OlTDBgwAGZmWb9WbW1t0bNnT5w9exYfP35UeX0/Pz+8ePECu3btAsB3Ha9atQrfffcdXF1dNYqVEErsCNGj0qVLK2zLBo5/+vQJAD9+jTEGJycnmJubK3ydPXtW3g2XmZmJ9u3bIyoqCpMmTcLRo0dx/vx5eSIlu96XNJ3dOX/+fFy4cAHHjx9HQEAAnj9/ju7du8vHiL1+/RoSiQTLli3LFmvnzp0BQB7v69ev4eTklO0eyvapivX58+e4du1atnsVK1YMjDH5vVq2bIndu3dDIpFg4MCBEAqFqFmzJsLDw+XXmjp1Kn7//XecPXsWnTp1QunSpdGmTZscy7m8fv1aZWxly5aVH5exsbHJ1oVmaWmZ63gpTe+j7DxVr3eZMmVyvPeXvv6sAnz8X362Fi9ejJ9++gmNGzdGZGQkzp49iwsXLqBjx44afQb9/Pxw7949HD58GACwYsUKNG3aFPXq1cs1zqFDh+LMmTP477//AAAbN26EpaWlwh8NeXm/v6TsdStTpoz8vcjtPcvMzMTbt29VXr9u3bpwd3eXj03dt28f4uLick1sCVGGZsUSYkTs7e3BcRxiYmKUzhaU7btx4wauXr2KTZs2YdCgQfLj9+/fV3ltTWuVubm5ySdMtGzZEtbW1pg+fTqWLVuGCRMmoGTJkhAIBBgwYAB+/vlnpdeoUKECAD5J+HrSBQClg/lVxWpvbw9ra+tsA/i/PC7j5eUFLy8vpKWl4ezZswgODoavry9cXV3RtGlTFClSBOPGjcO4cePw7t07HDlyBNOmTUOHDh0QHx+vdFapLNFJTEzMduzp06cK98+PL+/z9WxVZfdR9lqVLl0a58+fz7Zf1eudV9u2bUOrVq2watUqhf0pKSlKz1f1GWzdujVq1qyJ5cuXw9bWFv/++y+2bdumVgx9+/bFuHHjsGnTJgQFBWHr1q3o3r07SpYsKT8nL+/3l5S9bs+ePZO/V7l9NszMzBTiUWbMmDH4/vvv8e+//2L58uWoXLky2rVrl+vzJ+Rr1GJHiBHp0qULGGNISEhAgwYNsn3VqlULQNYvyK+Tv68HYGvTpEmTULFiRcybNw8pKSmwsbGBp6cnLl++jNq1ayuNV/YLz8PDAzdu3MCtW7cUrrl9+3a179+lSxc8ePAApUuXVnovZV1WlpaW8PDwkE9YUDaT0c7ODj4+Pvj555/x5s0blYVjmzZtCmtr62wJx5MnT3Ds2DG0adNG7eeSE1m36tf3uXDhAm7fvq3WfTw9PZGSkoI9e/Yo7A8LC9NKjDIcx2X7DF67di3XmaDKjBkzBvv378fUqVPh5OSE77//Xq3HlSxZEt27d8eWLVuwb98+PHv2TOWEC0D99/tL4eHh8m5iAHj06BFOnz6NVq1aAQCqVKmCcuXKISwsTOG8Dx8+IDIyUj5TNic9evRA+fLlMX78eBw5cgSjRo2iwtEkT6jFjpA8OHbsmNJfCLIuyLxq3rw5Ro4ciSFDhuDixYto2bIlihYtisTERJw8eRK1atXCTz/9hKpVq+Kbb77BlClTwBhDqVKlsHfvXnlXli6Ym5tj7ty56NWrF0JDQzF9+nSEhoaiRYsWcHd3x08//QRXV1ekpKTg/v372Lt3L44dOwYA8Pf3xx9//IFOnTrh119/hZOTE8LCwuTdZ1+OS1LF398fkZGRaNmyJcaOHYvatWsjMzMTjx8/xqFDhzB+/Hg0btwYM2fOxJMnT9CmTRsIhUK8e/cOoaGhMDc3h4eHBwCga9euqFmzJho0aAAHBwc8evQIISEhcHFxUVl82c7ODjNmzMC0adMwcOBA9O3bF69fv8acOXNgZWWFWbNmaeV1rlKlCkaOHIlly5bBzMwMnTp1QlxcHGbMmAGRSJRtFrUyAwcOxJIlSzBw4EAEBQWhUqVK+Pvvv3Hw4EGtxCjTpUsX/Pbbb5g1a5Z81vOvv/6KChUqKMxCVkf//v0xdepUnDhxAtOnT4eFhYXajx06dCh27NiB0aNHQygUom3btgrH8/J+f+nFixfo0aMHRowYgaSkJMyaNQtWVlaYOnUqAP7zu2DBAvTr1w9dunTBDz/8gLS0NCxcuBDv3r3DvHnzcr2HQCDAzz//jMmTJ6No0aLZxlgSojYDTtwgxOTIZrSp+oqNjc1xVuzLly+VXu/rmW9//PEHa9y4MStatCiztrZm33zzDRs4cCC7ePGi/Jxbt26xdu3asWLFirGSJUuy77//nj1+/JgBYLNmzcr13qrIZgLu2rVL6fHGjRuzkiVLymedxsbGsqFDh7Jy5coxc3Nz5uDgwJo1a8YCAwMVHnfjxg3Wtm1bZmVlxUqVKsWGDRvGNm/ezACwq1evys/z8PBgNWrUUHrv9+/fs+nTp7MqVaowCwsLVqJECVarVi02duxY9uzZM8YYY/v27WOdOnVi5cqVYxYWFszR0ZF17tyZxcTEyK+zaNEi1qxZM2Zvb88sLCxY+fLl2bBhw1hcXJz8HFXvzfr161nt2rXl9/fy8so223fQoEGsaNGi2eJXNgNVGalUyubPn88qV67MzM3Nmb29Pevfvz+Lj49XOC+n1+rJkyesZ8+ezNbWlhUrVoz17NmTnT59Wu1Zsd999122a3p4eDAPDw/5dlpaGpswYQIrV64cs7KyYvXq1WO7d+9mgwYNYi4uLvLzZD8TCxcuzPF5Dx48mBUpUoQ9efIkx/O+JpVKmUgkYgBYQEBAtuPqvN/KyH4Wtm7dysaMGcMcHByYpaUlc3d3V/hZlNm9ezdr3Lgxs7KyYkWLFmVt2rRhp06dUjhH1eeKMX62LQD2448/avT8CfkSx9gX7caEEKJHI0eORHh4OF6/fq1RCw0peNLT0+Hq6ooWLVpg586dhg4HAF+g2NPTE7t27YKPj4/O77ds2TKMGTMGN27cQI0aNXR+P1IwUVcsIUQvfv31V5QtWxZubm54//499u3bh/Xr12vc7UYKlpcvX+LOnTvYuHEjnj9/jilTphg6JL27fPkyYmNj8euvv8LLy4uSOpIvlNgRQvTC3NwcCxcuxJMnTyCRSFCpUiUsXrwYfn5+hg6NGND+/fsxZMgQODs7Y+XKlWqVOCloevTogWfPnsHd3T1Pq8IQ8iXqiiWEEEIIKSCo3AkhhBBCSAFBiR0hhBBCSAFBiR0hhBBCSAFBkydykZmZiadPn6JYsWJUBZwQQgghescYQ0pKCsqWLZtrQXdK7HLx9OlTiEQiQ4dBCCGEkEIuPj4+2xrSX6PELhfFihUDwL+YxYsXN3A0hBBCCClskpOTIRKJ5DlJTiixy4Ws+7V48eKU2BFCCCHEYNQZEkaTJwghhBBCCghK7AghhBBCCghK7AghhBBCCggaY6clUqkUGRkZhg6DEKNjYWGR6/R8Qggh2kGJXT4xxvDs2TO8e/fO0KEQYpTMzMxQoUIFWFhYGDoUQggp8CixyydZUufo6AgbGxsqYkzIF2QFvhMTE1G+fHn6+SCEEB2jxC4fpFKpPKkrXbq0ocMhxCg5ODjg6dOnkEgkMDc3N3Q4hBBSoNHAl3yQjamzsbExcCSEGC9ZF6xUKjVwJIQQUvBRYqcF1L1EiGr080EIIfpDiR0hhBBCSAFBiV0h1KpVK/j7+8u3XV1dERISYrB4ChN6rQkhhiaVShEdHY3w8HBER0fTMIkCxqQSuxMnTqBr164oW7YsOI7D7t27czw/OjoaHMdl+/rvv//0E7CJuHDhAkaOHGnoMAoFeq0JIYYUFRUFV1dXeHp6wtfXF56ennB1dUVUVJShQyNaYlKJ3YcPH/Dtt99i+fLlGj3uzp07SExMlH9VqlRJRxGaJgcHB6OYAKJugWddFoLWdZFpY3mtCSGFT1RUFHx8fPDkyROF/QkJCfDx8aHkroAwqcSuU6dOCAwMhLe3t0aPc3R0RJkyZeRfAoFARxGapq+7BzmOw/r169GjRw/Y2NigUqVK2LNnj8Jjbt26hc6dO8PW1hZOTk4YMGAAXr16JT9+4MABtGjRAnZ2dihdujS6dOmCBw8eyI/HxcWB4zjs3LkTrVq1gpWVFbZt26Y0Po7jsHr1anh5eaFo0aIIDAwEAOzduxf169eHlZUV3NzcMGfOHEgkEvnj/vvvP7Ro0QJWVlaoXr06jhw5otDSm1MMGzduRLVq1WBlZYWqVati5cqV8uump6dj9OjRcHZ2hpWVFVxdXREcHCw/Pnv2bJQvXx6WlpYoW7YsxowZo/K1fvz4Mby8vGBra4vixYujV69eeP78ucK16tSpg61bt8LV1RUlSpRAnz59kJKSovL9JISQr0mlUvj5+YExlu2YbJ+/vz91yxYAJpXY5VXdunXh7OyMNm3aQCwWGzockzBnzhz06tUL165dQ+fOndGvXz+8efMGAJCYmAgPDw/UqVMHFy9exIEDB/D8+XP06tVL/vgPHz5g3LhxuHDhAo4ePQozMzP06NEDmZmZCveZPHkyxowZg9u3b6NDhw4q45k1axa8vLxw/fp1DB06FAcPHkT//v0xZswY3Lp1C2vWrMGmTZsQFBQEgC+M2717d9jY2ODcuXNYu3YtAgIClF776xjWrVuHgIAABAUF4fbt25g7dy5mzJiBzZs3AwCWLl2KPXv2YOfOnbhz5w62bdsGV1dXAEBERASWLFmCNWvW4N69e9i9ezdq1aql9L6MMXTv3h1v3rzB8ePHcfjwYTx48AC9e/dWOO/BgwfYvXs39u3bh3379uH48eOYN29eDu8eIYQoiomJydZS9yXGGOLj4xETE6PHqIguFOgCxc7Ozli7di3q16+PtLQ0bN26FW3atEF0dDRatmyp9DFpaWlIS0uTbycnJ2t83wZrG+DZ+2d5jjsvytiWwcWRF7V2vcGDB6Nv374AgLlz52LZsmU4f/48OnbsiFWrVqFevXqYO3eu/Pw//vgDIpEId+/eReXKldGzZ0+F623YsAGOjo64desWatasKd/v7++vVgusr68vhg4dKt8eMGAApkyZgkGDBgEA3Nzc8Ntvv2HSpEmYNWsWDh06hAcPHiA6OhplypQBAAQFBaFdu3bZrv11DL/99hsWLVok31ehQgV58jho0CA8fvwYlSpVQosWLcBxHFxcXOSPffz4McqUKYO2bdvC3Nwc5cuXR6NGjZQ+pyNHjuDatWuIjY2FSCQCAGzduhU1atTAhQsX0LBhQwB8krpp0yYUK1ZM/tyPHj0qT2IJISQ3iYmJWj2PGK8CndhVqVIFVapUkW83bdoU8fHx+P3331UmdsHBwZgzZ06+7vvs/TMkpCTk6xqGVrt2bfn3RYsWRbFixfDixQsAwKVLlyAWi2Fra5vtcQ8ePEDlypXx4MEDzJgxA2fPnsWrV6/kLXWPHz9WSOwaNGigVjxfn3fp0iVcuHBBIbmRSqVITU3Fx48fcefOHYhEInlSB0BlgvXltV++fIn4+HgMGzYMI0aMkO+XSCQoUaIEAD7pbdeuHapUqYKOHTuiS5cuaN++PQDg+++/R0hICNzc3NCxY0d07twZXbt2RZEi2X/Ubt++DZFIJE/qAKB69eqws7PD7du35Ymdq6urPKkD+D9YZO8FIYSow9nZWavnEeNVoBM7ZZo0aaJyLBcATJ06FePGjZNvJycnK/ziVUcZ2zK5n6Rl2r7n10s/cRwnT84yMzPRtWtXzJ8/P9vjZP8pdO3aFSKRCOvWrUPZsmWRmZmJmjVrIj09XeH8okWLqhXP1+dlZmZizpw5Slv7rKyswBhTuzDul9eWPcd169ahcePGCufJxmbWq1cPsbGx+Oeff3DkyBH06tULbdu2RUREBEQiEe7cuYPDhw/jyJEjGDVqFBYuXIjjx49ne01Vxfj1/pzeC0IIUYe7uzuEQiESEhKUjrPjOA5CoRDu7u4GiI5oU6FL7C5fvpzjXySWlpawtLTM1z202SVqjOrVq4fIyEi4uroqbYl6/fo1bt++jTVr1sj/kzh58qTWY7hz5w4qVqyo9HjVqlXx+PFjPH/+HE5OTgD4UiO5cXJyQrly5fDw4UP069dP5XnFixdH79690bt3b/j4+KBjx4548+YNSpUqBWtra3Tr1g3dunXDzz//jKpVq+L69euoV6+ewjWqV6+Ox48fIz4+Xv7Hw61bt5CUlIRq1aqp+1IQQkiuBAIBQkND4ePjA47jFJI72R+SISEhNLmwADCpxO79+/e4f/++fDs2NhZXrlxBqVKlUL58eUydOhUJCQnYsmULAP5D6urqiho1aiA9PR3btm1DZGQkIiMjDfUUCoSff/4Z69atQ9++fTFx4kTY29vj/v372L59O9atW4eSJUuidOnSWLt2LZydnfH48WNMmTJFqzHMnDkTXbp0gUgkwvfffw8zMzNcu3YN169fR2BgINq1a4dvvvkGgwYNwoIFC5CSkiKfPJFbS97s2bMxZswYFC9eHJ06dUJaWhouXryIt2/fYty4cViyZAmcnZ1Rp04dmJmZYdeuXShTpgzs7OywadMmSKVSNG7cGDY2Nti6dSusra0VxuHJtG3bFrVr10a/fv0QEhICiUSCUaNGwcPDQ+0uakIIUZe3tzciIiLg5+enMJFCKBQiJCRE44oTxDiZ1KzYixcvom7duqhbty4AYNy4cahbty5mzpwJgB/0+fjxY/n56enpmDBhAmrXrg13d3ecPHkS+/fvpw9vPpUtWxanTp2CVCpFhw4dULNmTfj5+aFEiRIwMzODmZkZtm/fjkuXLqFmzZoYO3YsFi5cqNUYOnTogH379uHw4cNo2LAhmjRpgsWLF8sTKIFAgN27d+P9+/do2LAhhg8fjunTpwPgu2pzMnz4cKxfvx6bNm1CrVq14OHhgU2bNqFChQoAAFtbW8yfPx8NGjRAw4YNERcXh7///htmZmaws7PDunXr0Lx5c9SuXRtHjx7F3r17Ubp06Wz3kZVeKVmyJFq2bIm2bdvCzc0NO3bs0OprRQghMt7e3oiLi4NYLEZYWBjEYjFiY2Pp92IBwjFlne1ELjk5GSVKlEBSUhKKFy+ucCw1NRWxsbGoUKFCrskCMbxTp06hRYsWuH//Pr755htDh1No0M8JIYTkT065yNdMqiuWEE3873//g62tLSpVqoT79+/Dz88PzZs3p6SOEEJIgUWJHSmwUlJSMGnSJMTHx8Pe3h5t27bFokWLDB0WIeQLUqkUMTExSExMhLOzM9zd3WkAPyH5QIkdKbAGDhyIgQMHGjoMQshXZMncX3/9hW3btiksRygUChEaGkpjvgjJI0rsCCGE6E1UVFS2WZlfki1IHxERQckdIXlgUrNiCSGEmK6oqCj4+PjkumYpQAvSE5JXlNgRQgjROalUCj8/P6WrHnyNFqQnJO8osSOEEKJzMTExObbUKUML0hOiOUrsCCGE6FxekjRakJ4QzdHkCUIIITqnSZJGC9ITknfUYkc0Fh0dDY7j8O7dO0OHkqtNmzbBzs5Oo8e4uroiJCREJ/EYm9mzZ6NOnTqGDoMUAu7u7hAKhbmu1SxDC9ITkjeU2BVCgwcPBsdx4DgO5ubmcHNzw4QJE/DhwwdDh6Z1vXv3xt27dw0dhtGaMGECjh49augwSCEgEAgQGhoKADkmdyKRiEqdEJIP1BVbSHXs2BEbN25ERkYGYmJiMHz4cHz48AGrVq0ydGhaZW1tDWtra53eIz09HRYWFiZ3bQCwtbWFra2tzq5PyJe8vb0RERGRrY6dg4MD+vXrBy8vr0K98gStwkG0gVrsCilLS0uUKVMGIpEIvr6+6NevH3bv3g0ASEtLw5gxY+Do6AgrKyu0aNECFy5cUHqdDx8+oHjx4oiIiFDYv3fvXhQtWhQpKSmIi4sDx3GIioqCp6cnbGxs8O233+LMmTMKj4mMjESNGjVgaWkJV1fXbMt/ubq6IjAwEAMHDoStrS1cXFzw119/4eXLl/Dy8oKtrS1q1aqFixcvyh/zdVfsgwcP4OXlBScnJ9ja2qJhw4Y4cuSIRq/d4MGD0b17dwQHB6Ns2bKoXLkyAL6wau/evVGyZEmULl0aXl5eiIuLkz9OIpFgzJgxsLOzQ+nSpTF58mQMGjQI3bt3l5/TqlUrjB49GuPGjYO9vT3atWsHALh16xY6d+4MW1tbODk5YcCAAQrV+iMiIlCrVi1YW1ujdOnSaNu2rbwFNjo6Go0aNULRokVhZ2eH5s2b49GjRwCyd8VmZmbi119/hVAohKWlJerUqYMDBw7Ij6v7XhKiire3N+Li4iAWixEWFgaxWIzExEQsWbIErVq1KrSJTFRUFFxdXeHp6QlfX194enrC1dUVUVFRhg6NmBhK7AgAvmUrIyMDADBp0iRERkZi8+bN+Pfff1GxYkV06NABb968yfa4okWLok+fPti4caPC/o0bN8LHxwfFihWT7wsICMCECRNw5coVVK5cGX379oVEIgEAXLp0Cb169UKfPn1w/fp1zJ49GzNmzMCmTZsUrrtkyRI0b94cly9fxnfffYcBAwZg4MCB6N+/vzzWgQMHqqyV9f79e3Tu3BlHjhzB5cuX0aFDB3Tt2hWPHz/W6PU6evQobt++jcOHD2Pfvn34+PEjPD09YWtrixMnTuDkyZOwtbVFx44dkZ6eDgCYP38+/vzzT2zcuBGnTp1CcnKyPJn+0ubNm1GkSBGcOnUKa9asQWJiIjw8PFCnTh1cvHgRBw4cwPPnz9GrVy8A/GzDvn37YujQobh9+zaio6Ph7e0NxhgkEgm6d+8ODw8PXLt2DWfOnMHIkSNVdoWFhoZi0aJF+P3333Ht2jV06NAB3bp1w7179xTOy+m9JCQ3AoEArVq1Qt++fQt1MgfwrXS//vorevbsma0cjGwVDkruiEYYyVFSUhIDwJKSkrId+/TpE7t16xb79OmT4oH69RkrV06/X/Xrq/2cBg0axLy8vOTb586dY6VLl2a9evVi79+/Z+bm5uzPP/+UH09PT2dly5ZlCxYsYIwxJhaLGQD29u1b+eMFAgFLSEhgjDH28uVLZm5uzqKjoxljjMXGxjIAbP369fJr3rx5kwFgt2/fZowx5uvry9q1a6cQ58SJE1n16tXl2y4uLqx///7y7cTERAaAzZgxQ77vzJkzDABLTExkjDG2ceNGVqJEiRxfj+rVq7Nly5Yp3GfJkiUqzx80aBBzcnJiaWlp8n0bNmxgVapUYZmZmfJ9aWlpzNramh08eJAxxpiTkxNbuHCh/LhEImHly5dXeC88PDxYnTp1FO43Y8YM1r59e4V98fHxDAC7c+cOu3TpEgPA4uLissX6+vVrBkD+Xnxt1qxZ7Ntvv5Vvly1blgUFBSmc07BhQzZq1CjGmHrv5ddU/pwQUshFRkYyoVDIAKj84jiOiUQiJpFIDB0uMaCccpGv0Rg7XXj2DEhIMHQUOdq3bx9sbW0hkUiQkZEBLy8vLFu2DA8ePEBGRgaaN28uP9fc3ByNGjXC7du3lV6rUaNGqFGjBrZs2YIpU6Zg69atKF++PFq2bKlwXu3ateXfy0ofvHjxAlWrVsXt27fh5eWlcH7z5s0REhICqVQq/4v+y2s4OTkBAGrVqpVt34sXL1CmTJlssX748AFz5szBvn378PTpU0gkEnz69EnjFrtatWopjH27dOkS7t+/r9BCCQCpqal48OABkpKS8Pz5czRq1Eh+TCAQoH79+sjMzFR4TIMGDRS2L126BLFYrHQs3IMHD9C+fXu0adMGtWrVQocOHdC+fXv4+PigZMmSKFWqFAYPHowOHTqgXbt2aNu2LXr16qW09ERycjKePn2q8N4D/Ptw9epVhX05vZeEkNzJlldjuazEwb5YhaNVq1b6CY6YNErsdEFJQmFs9/T09MSqVatgbm6OsmXLwtzcHEBWEdGvu+oYYznOZBs+fDiWL1+OKVOmYOPGjRgyZEi282X3+PL6sqRG2fWV/Yen7Bo5XfdrEydOxMGDB/H777+jYsWKsLa2ho+Pj7y7VF1FixZV2M7MzET9+vXx559/ZjvXwcEhW3wyyp6jsmt37doV8+fPz3aus7MzBAIBDh8+jNOnT+PQoUNYtmwZAgICcO7cOVSoUAEbN27EmDFjcODAAezYsQPTp0/H4cOH0aRJE6XPTZ33XpPXnBCiSJPl1WRoFQ6iLkrsdOGLwfvGqmjRoqhYsWK2/RUrVoSFhQVOnjwJX19fAEBGRgYuXrwIf39/ldfr378/Jk2ahKVLl+LmzZsYNGiQRvFUr14dJ0+eVNh3+vRpVK5cWavjb2JiYjB48GD06NEDAD/m7ssJDnlVr1497NixA46OjihevLjSc5ycnHD+/Hl50VWpVIrLly/nWkeuXr16iIyMhKurK4oUUf4jy3EcmjdvjubNm2PmzJlwcXHB//73P4wbNw4AULduXdStWxdTp05F06ZNERYWli2xK168OMqWLYuTJ08qtLaePn1aoaWREJI/eVlejVbhIOqiyRNEQdGiRfHTTz9h4sSJOHDgAG7duoURI0bg48ePGDZsmMrHlSxZEt7e3pg4cSLat28PoVCo0X3Hjx+Po0eP4rfffsPdu3exefNmLF++HBMmTMjvU1JQsWJFREVF4cqVK7h69Sp8fX210tLUr18/2Nvbw8vLCzExMYiNjcXx48cVyjr88ssvCA4Oxl9//YU7d+7Az88Pb9++zbVg688//4w3b96gb9++OH/+PB4+fIhDhw5h6NChkEqlOHfuHObOnYuLFy/i8ePHiIqKwsuXL1GtWjXExsZi6tSpOHPmDB49eoRDhw7h7t27qFatmtJ7TZw4EfPnz8eOHTtw584dTJkyBVeuXIGfn1++XyNCjJ1UKkV0dDTCw8MRHR0NqVSqk/to0vrGcRxEIhGtwkHURi12JJt58+YhMzMTAwYMQEpKCho0aICDBw+iZMmSOT5u2LBhCAsLw9ChQzW+Z7169bBz507MnDkTv/32G5ydnfHrr79i8ODBeXwWyi1ZsgRDhw5Fs2bNYG9vj8mTJyM5OTnf17WxscGJEycwefJkeHt7IyUlBeXKlUObNm3kLXiTJ0/Gs2fPMHDgQAgEAowcORIdOnTItUWybNmyOHXqFCZPnowOHTogLS0NLi4u6NixI8zMzFC8eHGcOHECISEhSE5OhouLCxYtWoROnTrh+fPn+O+//7B582a8fv0azs7OGD16NH744Qel9xozZgySk5Mxfvx4vHjxAtWrV8eePXtQqVKlfL9GhBizqKiobPX1hEIhQkNDtV4sWdPWN1qFg2iCY5p08hdCycnJKFGiBJKSkrJ1saWmpiI2NhYVKlSAlZWVgSI0Hn/++Sf8/Pzw9OlTnRbVLSgyMzNRrVo19OrVC7/99puhw9EZ+jkhxk7VRAZZa7q2V8KQSqVwdXVFQkJCjuPsdJVYEtOTUy7yNeqKJfn28eNH3Lx5E8HBwfjhhx8oqVPh0aNHWLduHe7evYvr16/jp59+QmxsrHwsIyFE/3KayCDb5+/vr9VuWXWWV5szZw7i4uIoqSMao8SO5NuCBQtQp04dODk5YerUqYYOx2iZmZlh06ZNaNiwIZo3b47r16/jyJEjKse7EUJ0L7eJDF+WG9Em2fJq5cqVU9gvEokQGRmJmTNnUvcryRMaY0fybfbs2Zg9e7ahwzB6IpEIp06dMnQYhJAvqDuRQRflRry9veUTrmh9WKItlNgRQggptNSdyKCrciOy5dUI0RbqiiWEEFJoubu7QygUqhzrRuVGiKmhxE4LqOI+IarRxHuiD3mtQZfTRAbZNpUbIaaEumLzwcLCAmZmZnj69CkcHBxgYWGRa7FZQgoTxhhevnwJjuMUliEjRJvyW4NONpFB2TVCQkJoZioxKVTHLhe51Y5JT09HYmIiPn78aIDoCDF+HMdBKBTC1tbW0KGQAkibNeikUilNZCBGSZM6dpTY5UKdF5MxBolEorPlZwgxZebm5vTLkeiErNCvqnIlsj8qYmNj6TNITJomiR11xWqBrJuJupoIIUR/NKlBRzNPSWFBkycIIYSYJEPWoCPEWFFiRwghxCQZugYdIcaIEjtCCCEmiWrQEZIdJXaEEEJMEtWgIyQ7SuwIIYSYLFkNunLlyinsFwqFGpU6IaSgoHInudBkijEhhBDDoBp0pCCjcieEEEIKFYFAQCVNCAF1xRJCCCGEFBiU2BFCCCGEFBCU2BFCCCGEFBCU2BFCCCGEFBCU2BFCCCGE5NHrj6+x6PQiSDOlhg4FAM2KJYQQQgjR2LP3z7D4zGKsvLASHzI+QFRChF41ehk6LErsCCGEEELUFZ8UjwWnFmD95fVIlaTK9y88vZASO0IIIfpBBXyNG70/xu/BmweYd3IeNl/djIzMDPl+S4ElhtcbjonNJhowuiyU2BFCSAEXFRUFPz8/PHnyRL5PKBQiNDS00Cy5ZcyJE70/xu3Wy1sIPhmMsOthyGSZ8v1FzYvixwY/YnzT8XAu5mzACBXRkmK5oCXFCCGmLCoqCj4+Pvj6v3qO4wCgUKynasyJE70/xuty4mUExQQh6nYUGLLen+KWxTGm0Rj4NfGDvY29XmLRJBehxC4XlNgRQkyVVCqFq6urQkLzJY7jIBQKERsbazStV9qmk8SJMeDlS+DePeDuXeDxY+DFC37fy5fAp09Aejr/lZkJ2NgA1tb8vw4OgLMzULYspGXLwtXPD0+eP1d6m8Lw/hijM/FnEBQThP339ivsL21dGmObjMXPjX6GnZWdXmOixE6LKLEjhJiq6OhoeHp65nqeWCwukOusai2xff4cOHUKuHABuHgRuHQJePs23/FFA8j93Sm4748xYYwhOi4agTGBOBZ7TOFYGdsymNhsIkbWHwlbC1uDxKdJLmJSdexOnDiBrl27omzZsuA4Drt37871McePH0f9+vVhZWUFNzc3rF69WveBEkKIEUhMTNTqeaYmJiZGZVIH8L/M4+PjERMTo3ggLQ345x9g3Digdm2gTBmgZ09g3jzgyBH1kjoLC8DWFihWDFCRNKr7qidevarmmURTjDH8c+8ftNjYAq23tFZI6sqXKI8VnVcg1i8W45qOM1hSpymTmjzx4cMHfPvttxgyZAh69uyZ6/mxsbHo3LkzRowYgW3btuHUqVMYNWoUHBwc1Ho8IYSYMmdn9QZ0q3uePmhzkoNGiW1qKnDwILBrF7B3L5CcrPoBZcrwCV+lSvyXmxu/z8GB/7KxAT539cplZADv3/Ndtk+fAk+fwvnYMeCPP3KNz9nfnz+vTx/+q0IFtZ4XUS2TZWL3f7sRFBOEfxP/VThWsVRFTGsxDf1q94OFwMJAEeadyXbFchyH//3vf+jevbvKcyZPnow9e/bg9u3b8n0//vgjrl69ijNnzqh1H+qKJYSYKllXZEJCQrYxZoDxjeHS9iQHtbuiu3ZFq+hoICUl+0GOAxo0AFq3Bpo0ARo2BMqV0zgWZXJ9fwAIAcQCUHh3mjQBRo4Eevfmk0iiNkmmBDtv7kRQTBBuvbylcKyGQw0EuAfg+xrfo4iZcbV7FdiuWE2dOXMG7du3V9jXoUMHXLx4ERkZGUofk5aWhuTkZIUvQggxRQKBAKGhoQCyJgvIyLZDQkKMJqnz8fHJ1nWakJAAHx8fREVFaXxNd3d3CIXCbM9dhgMgAuC+d69iUleiBDBwIBARAbx+DZw/z3fDdu+utaQOUOP94TiE+PpC0Lix4gPPngWGDgXKlgXGjOEncJAcpUvTseHfDai2ohr6RfVTSOrqO9fH/3r/D9d+uoa+tfoaXVKnqQKd2D179gxOTk4K+5ycnCCRSPDq1SuljwkODkaJEiXkXyKRSB+hEkKITnh7eyMiIgLlvkpIhEKh0ZTSkEql8PPzU9pqJdvn7+8PqVSztThzTJw+/xuCz61hRYsCgwYB+/fz3aWbN/Pj6kqWzDHu6OhohIeHIzo6WuP4ADXenz//5BO5hw+B4GCgVq2sk5KSgGXLgKpVAR8ffmIHUfAp4xNWnF+BiksrYvje4bj/5r78WDNRM/zT7x9cGHEB3at2hxlXMFKiAt0VW7lyZQwZMgRTp06V7zt16hRatGiBxMRElClTJttj0tLSkJaWJt9OTk6GSCSirlhCiEkz5gK96naZHjlyBAKBQOPnELVuHfzGjsWTDx/k+0Tgkzrvhg2BESP4sWvFiqkds7a7jdV+fxjjZ+euXg1s386XVvlSmzbAzJlAy5Yax1CQvE9/j9UXV2PRmUV49v6ZwrE2Fdpgesvp8HDxUNmaa2w06Yo17fbGXJQpUwbPnim+oS9evECRIkVQunRppY+xtLSEpaWlPsIjhBC9EQgERlsyQ91JDr169cKbN2/k27kmUq9eQRoUhFIrVmBeRgZeAnAAUM7MDO59+kAwYQJQt67G8aqqjSfrNs5LS6ja7w/HAY0a8V+LFwNr1wJLlgCy33VHj/JfnTvzLXy1a2sUh6l7l/oOy88vx5KzS/Dm0xuFY10qd0GAewCaCJsYKDr9KBjtjio0bdoUhw8fVth36NAhNGjQAObm5gaKihBCCg5tdEeqOyv3y6QOyGH83fv3wOzZiBKJ4BoSAs+MDPQHMBbAFFtbvFm5EoI//8xTUqerbuM8sbMDJk0CYmOBNWuAb77JOvb330CdOnz38uPHuo/FwF5+eImAowFwCXHBDPEMeVLHgYNPdR/8O/Jf7O27t8AndQAAZkJSUlLY5cuX2eXLlxkAtnjxYnb58mX26NEjxhhjU6ZMYQMGDJCf//DhQ2ZjY8PGjh3Lbt26xTZs2MDMzc1ZRESE2vdMSkpiAFhSUpLWnw8hhJiyyMhIJhQKGQD5l1AoZJGRkRpdRyKRMKFQyDiOU7iWOl8cxzGRSMQkEgljmZmMhYUxVq4ciwQYp+J8juM0jlFGLBarFZdYLM7T9fNFImFs40bGRCLG+E5b/svGhrHgYMbS0nRwSwkTi8UsLCyMicVi/n3Qo6fJT9m4A+OYTZANw2zIvwRzBGxA1AB268UtvcajK5rkIiaV2Kn6gRo0aBBjjLFBgwYxDw8PhcdER0ezunXrMgsLC+bq6spWrVql0T0psSOEkOwiIyOVJmJ5TZxk18tLcgeAidetY6xlS8YAJgGYUN1kUENhYWFqxRMWFqbxtbXm0yfGfv+dsZIlFRO8KlUYO3JEa7fRVmKfF3Fv49iofaOY5W+WCgmd+a/mbMSeEez+6/s6j0GfCmxiZwiU2BFCiCJZC1tOiY1QKNQ4cVKWKJQqVUq9RIrj5AmMWN1kMA+takbdYve1t28ZGzOGMTMzxQTP15exV6/ydWltJ/bquvvqLhuyewgr8msRhYTOKtCKjfl7DHv87rFO7mtomuQiBXqMHSGEEO3LbakuAHjy5AmCgoI0uq63tzfi4uIgFosRFhYGsViMnTt3qvVYZ9mYt4oVkThxolqPyctSarnWxuM4iEQiuLu7a3xtrbOzA0JD+bVtmzbN2h8WBtSoAfz1V54ua4hxhjde3IBvpC+qrqiKjVc2QpIpAQDYWthiUrNJiPOLQ2inUIhKUIkySuwIIYRoRN2EaNasWRoXFpbNDu3bty9atWqFVq1aqVdk2MICCAwEbtyAc+fOat0rL0upmVLRZ7k6dYCTJ/llyWR1+Z4/5wsu9+8PfDUpJTd5XoM3Dy4+vYgeO3qg1qpaCL8RjkyWCQCws7LDLI9ZeOT/CPPbzYeTrVMuV9KMNiYFGQoldoQQQjSiSUKU35YbtYoMV60KwdWrQEAAYGmp81Y1Uyj6nI2ZGTBkCHDzJtC1a9b+P/8Evv0WOHFC7UtptAZvHp16fAqd/uyEhusaYvd/u+X7HWwcENwmGI/8H2F2q9koZV0qz/dQJSoqCq6urvD09ISvry88PT3h6uqap9VPDIESO0IIIRqRJU7q0EbLjbe3NyJ27kS5rwoICzkOEcOHw/vmTX71hc/00aqmrNs4NjbWOJO6Lzk7812wmzfzXbUA8OQJ4OkJzJkDqJGEq5vYa9oiyhjDkYdH4LnZEy02tsCB+wfkx8oWK4uQDiF48MsDNJE0wf6o/TppSdPF0nZ6p+PxfiaPJk8QQkh2kZGRas9YzfcM0dhYxlq0YJLPEyPCACauVo1J7t7NNcavJ2OIRCK9zNo0CU+eMNaqleLEipYtGYuPz/FhuZWn0XTWcWZmJtt7Zy9rvK6xwoQIzAZzDXFlqy+sZqkZqTqfhZvbpKD8zKbOL5oVq0WU2BFCiHJz5szR2QxReX20iROZuGhRJpElHmZmjM2axVhGhmbXMVCdNaMnkTD222+KM2cdHRk7fjzHh6kqT6PJrFiJVMJ23tjJvl31bbaErsqyKmzT5U0sXZKucD9lyZa2ZuEa84xnSuy0iBI7QghRTiKRsHLlymm9hUNpywzAIh0cGDt5UkfPppCLiVEsbFykCGNLl/JFn1XIa4tohjSDbbmyhVVdXjVbQldrZS2248YOJpFmfWb01ZJmzDUKNclFCvRasYQQQnRHIBBg6dKl8PHxAQCF8hd5Hcumch1WAD6vXiHi+XMY+Sg2rZNKpYiJiUFiYiKcnZ3h7u6u/Vm3LVoAly8DffsChw8DEgkwZgxw8SKwejVgbZ3tId7e3vDy8lI7tjRJGjZf3Yz5p+bj4duHCscalm2I6S2no0vlLjDjFIf/azILNz/rIetq7KDe6TrLNHXUYkcIKYi02UWprbFsEomECZ2cjHKMk6HofXUHiYSxyZMVx93Vr8+Px8ujD+kfWOjZUFZuUblsLXQtN7Zkh+4fYpk5tAzqqyVN22MHtYm6YrWIEjtCSEGji2RBG4mi+LffjHaMkyEYanUHxhhjO3bwa8zKkrty5Ri7ckWjSySnJrP5J+czx4WO2RK69lvbs+NxOY/jk9Hn2DdtjB3UBUrstIgSO0JIQWLQZEGVzEzGgoNZmBq/vLXRMmMKjGKG5rVrjFWokJXc2doy9s8/uT7szcc3bLZ4Nis5r2S2hM4r3Iudf3JeozD03ZJmjLOpNclFOMaUrAlC5JKTk1GiRAkkJSWhePHihg6HEELyTCqVwtXVVeV4JY7jIBQKERsbq7+VE9LSgJEjgS1bEA3AU42HiMXifI2lMgXR0dHw9Mz91dD5a/HiBdCtG3DuHL8tEAArVgA//JD91A8vsOTMEqy4sAIp6Sny/Rw49KrRC9Pcp6G2U+08hSEbewkoH8up7cLQehnXqAFNchGaPEEIIYWEvgahq+3tW8DLC/hcwNgdgLB4cSSkpChdh1SWeDZr1gzR0dFG80tXF/SxuoNaHB2BY8eAgQOByEi+gPGPPwKPHgFBQQDHISE5AQtPL8TaS2vxSfJJ/lABJ8CAbwdgSvMpqGJfJV9hyFb78PPzU/gMC4VChISEaL0wtGxpO1NEiR0hhBQSRpMsAPxqB506ATdu8NvW1hBs3YpQjoOPjw84jlPaMtOnTx9888032X65h4aGGv+qDxowqhmaNjbAzp3A5MnA77/z+4KDkfw0DpN72OKP65uRLk2Xn24hsMDQOkMxqfkkVChZQWthaDoLt7CirthcUFcsIaSgMJruvVu3gI4dgfh4ftvREdi/H2jQAADf7fZ1y4xIJEKfPn3w+++/Z2vN01V3nCHJus0TEhJybL3Ua7c5AKxcCTZ6NLjPMe2sDgzwBtKLANZFrPFjgx8xvul4lCteLpcLEU1okotQYpcLSuwIIQWFUSQLp08DXbrw3bAA8M03wMGD/L9fxfply0yzZs2ytdTpPXY90/e4stxce34NQTFBMNu+E1v+B5hn8vuPVRTgxBJ/jPKcBMeijnqLpzChxE6LKLEjhOiLPgZsGzRZ2LsX6NULSE3lt+vVA/7+G3ByyvWhRtPaqGeqWi91Ma5MlfMJ5xEUE4Q9d/bI93W8B0TuBGwyPu9o0gT45x/Azk4vMRU2muQiZjkeJYQQohdRUVFwdXWFp6cnfH194enpCVdXV0RFRWn1PrJB6OXKKXaVCYVC3SZ1O3YA3t5ZSV27dkB0tFpJHWBk4wP1yNvbG3FxcRCLxQgLC4NYLEZsbGye3iepVIro6GiEh4cjOjoaUqk0x/NPPDqB9lvbo/H6xgpJnWNRR7T+aQFw8BBQogS/8+xZ/j2VtcQSg6EWu1xQix0hRNdULaOly1Y0vZZz2LwZGDoUyPzcd9e3L7BpE2BhofYlCmuLnbYoa/lTNumEMYZDDw4hKCYIMY9jFK4hLC7EpGaTMLzecFibf15i7OpVoH17viwKwLfCHj4MlCql8+dUmFBXrBZRYkcI0SWjrC2nTatXAz/9lLU9YgS/z0yzDiN9jA80ttpl2qLOHw7de3TH3jt7ERgTiItPLyqc51bSDVNbTMXAbwfCQqAkGb91C2jdGnj+nN+uW5dP7kqX1snzKYyoK5YQQkyEJrXlTE5IiGJSN2YMsGaNxkkdwNcVCw0NBZCVkMjItkNCQvKciOmrK1zfpFIp/Pz8lCbDsn0jfx6Jb1d+i+47uiskddXsq2Frj624M/oOhtcbrjypA4Dq1QGxGChTht++fBlo0wZ49Urrz4fkjhI7QggxIFMaO6bRGK1584CxY7O2J0/mE72vkjJN6Gp8oKxF6+sEOyEhAT4+Piad3Knzh8PrZ69x48IN+b46Zeog4vsI3Bh1A/1r90cRMzVK3larxo+ZlNXVu3qVT+5ev87nMyCaogLFhBBiQEZViDYH6o7RAsAXsZ06NWt7zhxgxox8JXUy2i5Sm1uLFsdx8Pf3h5eXl0l2y6r9B8F7oImwCaa7T0fnSp2ztYqqpUoVPrnz9ASePgWuXePrFR49CtBQJv3RxuK0BZkmC+8SQoim9L3AeV5ERkYqjY/jOMZxnOLi6KGhWYvGA4zNm2ewuNUhFouVvu5ff4nFYkOHmifqPr9FYYtYZmamdm569y5jzs5ZnwF3d8Y+fNDOtQspTXIR6oolhBAD0vXYsfxSZ4yWv78/3y27ahXg55d1QmAg3wVrxEypKzwvvm30LYo75NBaxvF18fx6+eWtlU6ZSpUUJ0/ExPClbtLStHN9kiNK7AghxMAMVltODWpP7pgyBRg1KuvAzJlAQIAeIswfU+kK19Trj68xUzwTbsvckOyZrPQcjuPAgdPNHw41avArisi6YA8e5MvcSCTavQ/Jhsqd5ILKnRBC9MUYy22Eh4fD19c31/PCAPSVbUyZAsydq5UxdbpmFMusadGz98+w+MxirLywEh8yPsj3c7c5WB+xxsfXH+X79LKCxcmTfJ27T5/47f79+bqGeZgZXZhpkovQ5AlCCDESAoHA6Irrqt2iJftm3DiTSeqArK5wHx8fcByndJk1Q3aFqys+KR4LTi3A+svrkSpJle8vYlYEA2sPxJTRU+Bm56b/PxxatAD++otfHzg9Hdi2DbC3BxYvNpnPiKmhFrtcUIsdIaQwy7VFC4AQQCwAwejRwNKlJvkL2xjWZM2LB28eYN7Jedh8dTMyMjPk+y0FlhhebzgmNpsIFzsXA0b42V9/AT17ArISOQsXAhMmGDYmE0IrT2gRJXaEkMJOVucNgGKL1ud/IwB4Dx4M/PGHSSZ1MsbYFa7KrZe3EHwyGGHXw5DJMuX7i5oXxY8NfsT4puPhXMzIxgVu2AAMH561vXUr3zWrZab0PqqLEjstosSOEEJUtGgBCAHg7eUFREQARWh0j65dTryMoJggRN2OAkPWr+/ilsUxptEY+DXxg72NvQEjzEVgIF/TEOA/L/v382PwtESjeosmhBI7LaLEjhBCeFKpFDEREUgcNQrOb97AHYDAwwM4cACwsjJ0eAXamfgzCIoJwv57+xX2l7YujbFNxuLnRj/DzsrOMMGpSSqVIubECSTOnQvnI0f4z0/RonxR4wYN8n19ddbENdXkjhI7LaLEjhBCPnv5EnB3B+7c4bfr1ePXCKX/G3WCMYbouGgExgTiWOwxhWNlbMtgYrOJGFl/JGwtbA0UofqUtqQBCAXg7eAAnD4NVKyo1rWUdbUCgKurq8rSPKY2u/lrNCuWEEKIdn34wM9slCV1lSoB//xDSZ0OMMZw4P4BBMYE4nT8aYVj5UuUx+TmkzG07lBYFcm9ldQYxpupaklLAOADIOLlS3h37gycPQuUKpXrtZR1tY4YMUK9eosxMUY381zbqMUuF9RiRwgp9KRSfuWAPXv47bJlgVOnAFdXg4ZV0GSyTOz+bzeCYoLwb+K/CscqlqqIaS2moV/tfrAQWKh1PWMYbyabVa2yJQ1fzKr28AAOHQIslD+/nLpa1U1lwsLC0Ldv39xPNDKa5CJUIZAQQohqjPHLhMmSuuLF+TF1lNRpjSRTgrDrYai1qhZ67uypkNTVcKiBMO8w3P75NobUHaJRUufj45MtoUpISICPjw+ioqK0+hxUyXXlEgDxAGIA4PhxYORI/jP3FXWWtlOHqa0gkhfUFUsIIUS1xYuBFSv474sUASIjgVq1DBKKMXQralO6NB1br27FvFPzcP/NfYVj9Z3rI8A9AF5VvWDGadYGk1sSxHEc/P394eXlpfPXT+21eM3NgYwMflWKSpWyLUeXW4KYG9kYO9l4vIKMWuwIIYQot3OnYhHZ9euBtm0NEkpUVBRcXV3h6ekJX19feHp6wtXVVW8tT9r0KeMTVpxfgYpLK2L43uEKSV0zUTP80+8fXBhxAT2q9dA4qQM0WN83JiZP8WtC7ZVLvkzkpk8HduxQOK5ugghkzYL9etsUVhDRBkrsCCGEZBcTAwwYkLU9Zw4waJDK06VSKaKjoxEeHo7o6GhIZSsMaIGxdCvm1/v09/j99O9wW+qG0f+MRnxyvPxYmwptIB4kxskhJ9GxYsdsyYkm1G4l0yBZyit3d3cIhUKVz4fjOIhEIrhPnw4EB2cdGDQIOHNGvqlugjhnzhyUK1dOYZ9QKDTpUieaoskTuaDJE4QQQzBot+OdO0DTpsDbt/z2kCH8qgEqfjnrcpB+roPvTaCMxbvUd1h+fjmWnF2CN5/eKBzrUrkLAtwD0ETYRGv3i46OhqenZ67nicVivcwQVblyydf15RjjV6b44w/+BAcH4Nw5oEKF3Je2++JzAKBAddkDGuYijOQoKSmJAWBJSUmGDoUQUkhERkYyoVDIwI8tZwCYUChkkZGRur/5q1eMffMNY/yvWcbat2csPT3HWDmOU4gVAOM4jnEcl++YxWJxtmsr+xKLxfm6jy68eP+CTTsyjRUPLs4wG/IvbjbHfHb6sH+f/quT+0okEiYUCpW+L7L3RiQSMYlEopP7K6PsMy0SibJ/PtLSGPP0zPr81ajBWHKy/Bqyz5UuPmvGTJNchBK7XFBiRwjRJ10nSjlKS2PMwyPrl2rt2ozl8H+fLIFQlWxpI4EICwtTK7ELCwvL8z207WnyUzbuwDhmE2SjkNAJ5gjYgKgB7NaLWzqPwRiTIIlEwsRiMQsLC2NisVj15+LNG8aqVMn6HHbvzphUyhjTIEEsYDTJRagrNhfUFUsI0ReDdjsyBvzwA7BuHb/t5AScPw+UL6/yIfro8jO2bsWcPHr3CAtOLcCGyxuQJk2T7zc3M8fgOoMxuflkfFPqG73Fo3R9X5EIISEhxj/e7O5doFEjICmJ3545kx/niYI3O1odtPIEIYSYIE1mM2o9iVm6NCups7QEdu/OMakD9DNIXzb4PrexVYYsY3Hv9T0EnwzG1mtbIcmUyPdbFbHCyHojMaHZBIhKiPQel7e3N7y8vEwzCapcGdi+HfjuOyAzE/j1V6B2baBnTwgEAoMn8caMEjtCCDESBpvN+M8/wLhxWdsbNgBNch/Mr3Ypi3wUhRUIBAgNDYWPj0+2FQYMXcbixosbmBszFztu7kAmy5Tvt7WwxagGozCu6Tg42TrpPa4vmXQS1LEjMH8+MHEivz1wIF/jrnZtw8Zl5KjcCSGEGAl9JErZ3LoF9OnDt4oAwLRpQL9+aj1U7VIW+WxN8/b2RkREhNGUsbj49CJ67OiBWqtqIfxGuDyps7OywyyPWXjk/wjz2803eFJXIIwfD/Tvz3//8SPg5QW8emXYmIycxmPsWrdujaioKNjZ2SnsT05ORvfu3XHs2DFtxmdwNMaOEKIvmpR00EoL1atXQOPGwMOH/HaPHkBEBGCm/t/8apey0AJDj6069fgUAmMCceD+AYX99jb2GN90PEY1HIXilrr/PWHo10HvPn0CPDyACxf47Vat+DVlzc0NGpY+6bTcCcdx7Pnz59n2P3/+nBUpUkTTyxk9mhVLCNEnvc1m/HoGbJ06jL1/n+eYC+pMxczMTHb4wWHWalMrhRmumA1WdlFZtuTMEvY+LW+vW14YtBSOIT15wliZMlmf11GjDB2RXulkVuy1a9cAAHXq1MGxY8dQqlQp+TGpVIoDBw5gzZo1iIuL0ywNNXLUYkcI0Te9zGYcPTprDVgnJ741RJT3Af4FrRWJMYb99/Yj8EQgziWcUzjmaueKKc2nYHCdwbAsYqm3mGSto1//2tZF66hROnuWb7lLT+e3//iDL55dCGiSi6id2JmZmck/PMoeYm1tjWXLlmHo0KF5CNl4UWJHCDEEnSZKGzcCsv+rLSyA48fVmixRGEgzpYi6HYWgmCBcfX5V4ViV0lUwtcVU+NbyhblAv92ABWEFDq3YtCkrmbO0BE6dAurXN2hI+qCTxO7Ro0dgjMHNzQ3nz5+Hg4OD/JiFhQUcHR318mFauXIlFi5ciMTERNSoUQMhISEqB+aqqn90+/ZtVK1aVa37UWJHCClQLlwA3N2BtM911jZsyEryCjFJpgTh18Mx9+Rc/PfqP4VjtRxrYXrL6ehZrScEZoZJmkypnp/OjRoFrFrFf+/iAly6BJQubdiYdEwndexcXFwAAJmZmbmcqTs7duyAv78/Vq5ciebNm2PNmjXo1KkTbt26hfI51Fu6c+eOwgvxZVJKCCGFxosXgLd3VlL300+FPqlLk6Rh89XNmH9qPh6+fahwrGHZhpjecjq6VO4CM86wRSQMVgrHGIWEAJcv812zjx4BffvyJXsKckulBjSuY7dly5Ycjw8cODDPweRm8eLFGDZsGIYPHw6Ar1108OBBrFq1CsHBwSof5+jomG0WLyGEFCoZGcD33wOyrrzmzflfkIXUx4yPWP/veiw4tQAJKQkKx1q6tMR09+lo69ZWZSkXfTNIKRxjZWHBz96uV4//Y+XwYX5liqAgQ0dmFDQud1KyZEmF7YyMDHz8+BEWFhawsbHBmzdvtBqgTHp6OmxsbLBr1y706NFDvt/Pzw9XrlzB8ePHsz1G1nTt6uqK1NRUVK9eHdOnT8+xOTstLQ1paVlLwSQnJ0MkElFXLCHEpEl/+QUxy5cjEYBz6dJwv3oVgq/qwhUGKWkpWHlhJRafXYwXH14oHGv/TXsEuAegpUtLA0Wnmt5L4ZiC48eBNm0AqZTf3r2br3NXAGnSFatx2/Lbt28Vvt6/f487d+6gRYsWCA8Pz3PQuXn16hWkUimcnBQLPjo5OeHZs2dKH+Ps7Iy1a9ciMjISUVFRqFKlCtq0aYMTJ06ovE9wcDBKlCgh/xLlY5YYIYQYg6gxY+C6fDk8AfgC8Hz9Gq5NmiAqKsrQoenN209vMSd6DlxCXDDl6BSFpM6rihfODz+Pg/0PGmVSB2StwAEgWytiflbgkEqliI6ORnh4OKKjoyGVJUmmwMMDWLAga3vgQH6N2cJOWzVWLly4wKpUqaKty2WTkJDAALDTp08r7A8MDNTovl26dGFdu3ZVeTw1NZUlJSXJv+Lj46mOHSHEZEUuWMC4L2qeQVd18YzU8/fP2ZTDU1ixucUUatBxsznWe1dvdvXZVUOHqBFt1gwsEDXxMjMZ69Urq75d9eqMpaQYOiqt06SOndbWihUIBHj69Km2LpeNvb09BAJBtta5Fy9eZGvFy0mTJk2wbds2lcctLS1haam/ukSEEKIr0ufP4Td1KpSNt2GMgeM4+Pv7w8vLq8B13yUkJ2Dh6YVYe2ktPkk+yfcLOAH61+6PqS2moop9FQNGmDfe3t7w8vLKdykcVTXxEhIS4OPjYzo18TiOn9l94wa/PN6tW8CwYcD27fyxQkjjxG7Pnj0K24wxJCYmYvny5WjevLnWAvuahYUF6tevj8OHDyuMsTt8+DC8NOhTv3z5cuEYXEoIKdykUsR06YInOXStMcYQHx+PmJiYAlMiI/ZtLOafmo+NVzYiXZou328hsMDQOkMxqfkkVChZwYAR5p9AIMjX+yWVSuHn56d0rJ5JJvy2tkBUFNCwIZCSAuzcyddlHDvW0JEZhMaJXffu3RW2OY6Dg4MDWrdujUWLFmkrLqXGjRuHAQMGoEGDBmjatCnWrl2Lx48f48cffwQATJ06FQkJCfKZuyEhIXB1dUWNGjWQnp6Obdu2ITIyEpGRkTqNkxBCDC4wEIkXL6p1akEokfHfq/8QfDIYf177E1KWlcxaF7HGD/V/wIRmE1CueOGbLKJMTEyMykLHgIkm/FWqAFu28OsdA8CkSfw6yM2aGTYuA9A4sTNkHbvevXvj9evX+PXXX5GYmIiaNWvi77//ltfYS0xMxOPHj+Xnp6enY8KECUhISIC1tTVq1KiB/fv3o3PnzoZ6CoQQonuHDgFz5kDdvglT7sW49vwagmKCsOvmLrAvOp2LWRTD6Eaj4d/EH45FHQ0YofEpsDXxuncHJk8G5s8HJBKgd2++3p29vaEj0yuNy518SfZQY6nzowu08gQhxKTExwN16wKvX0MKwLV4cSSkpBS4EhnnE84jKCYIe+4oDg8qZV0Kfo398EujX1DSuqSKRxduBXoVC4mEL4Eiq37RoQPw99+AmWELTOeXTsudAMCGDRtQs2ZNWFlZwcrKCjVr1sT69evzFCwhhBAtSU8HevUCXr8GAAi6dEHohg0AtFsiw5BOPDqB9lvbo/H6xgpJnWNRRyxouwBxfnGY6TGTkrocuLu7QygUqmyU4TgOIpFI5XKdRq1IEX7ihOPnVtqDBwtd4WKNE7sZM2bAz88PXbt2xa5du7Br1y507doVY8eOxfTp03URIyGEEHVMnMgvswQArq7A5s3w/jzDsdxXxYiFQqHJzHxkjOHg/YNoubElPDZ54PDDw/JjwuJCLO24FHF+cZjYfCKKWRYzYKSmQVc18YyGszMQHp7VSjdrFnD0qGFj0iONu2Lt7e2xbNky9O3bV2F/eHg4fvnlF7x69UqrARoadcUSUrhJpdJ8l5bQi507+TFFAL/k0qlTQIMG8sMm8zy+kMkysffOXgTGBOLiU8WJIG4l3TC1xVQM/HYgLAQWBorQtEVFRcHPz09hIoVIJEJISIhJJPy5CgwEZszgv3d05MfblS1r2JjySJNcJE9Lip0/fx6VKlVS2H/37l00atQI79690zhgY0aJHSGFl7JffEKhEKGhocb1i++///hSD+/f89urVwM//GDYmPJBminFrlu7MDdmLq6/uK5wrJp9NUxzn4Y+NfugiJnWSrEWWqaY8KstMxP47jvgwAF+290dOHaM7641MTpN7H755ReYm5tj8eLFCvsnTJiAT58+YcWKFZpHbMQosSOkcFJVwFXWVWU03ZgfPvBlHW7e5Lf79+fLPpjgpLYMaQb+vP4ngk8G4+5rxaWh6pSpg+nu09GjWg+YcaY9EJ7o0atX/GQi2R9nkybxs2ZNjM4Tuy1btkAkEqFJkyYAgLNnzyI+Ph4DBw6Eubm5/Nyvkz9TRIkdIYWPbMF1VbW+jGY2KWP8+piy1XRq1ADOnQOKFjVcTHmQKknFpiubMO/kPDxKeqRwrHG5xpjRcgY6V+pcoCswEB06cwZo2ZKfMQsAf/0FdOtm2Jg0pNPETp0p0gD/H9+xY8c0ubRRosSOkMLHZMpBbNgADB/Of29rC1y8yBdqNREf0j9g7aW1+P3M73iaorgkpaerJwLcA9C6QmtK6Ej+hYRkrURhZwf8+y9QwXRWINEkF9G4o1ksFuc5MEIIMQUmUcD15k3gl1+yttevN5mkLjktGSvOr8Dis4vx6qPihLtOFTshwD0AzcvrbolKUgj5+QEnTwKRkcC7d3xZoJMngQK4NrzGAxWGDh2KlJSUbPs/fPiAoUOHaiUoQggxJHVXYjDYig0fP/K/mD59Xtz+xx+zZsQasdcfX2OmeCZcQlww7dg0haSuR9UeuDjiIv7u9zcldUT7OI5v4a5Ykd++eBEYN86wMemIxl2xAoEAiYmJcHRUXKLl1atXKFOmDCSyPuwCgrpiCSl8ZGPsEhISjHPFhuHD+V9SAFC7Nl+7ztpa/3Go6dn7Z1h8ZjFWXliJDxkf5PvNODP0qdkHU1tMRU3HmgaMkBQaV64ATZoAaWn8dng40KePQUNSh066YpOTk8EYA2MMKSkpsLKykh+TSqX4+++/syV7hBBiimQFXH18fMBxnEJyZ/ACrn/+mZXUFS0K7NhhtEldfFI8FpxagPWX1yNVkirfX8SsCAbWHogpLaagUulKOVyBEC2rUwdYvhwYMYLfHj6c31e1qiGj0iq1Ezs7OztwHAeO41C5cuVsxzmOw5w5c7QaHCGEGIq3tzciIiKU1rEzWAHXe/f4bleZlSv1/gtJnbpnD948wLyT87D56mZkZGbI91sKLDG83nBMbDYRLnYueo2bELlhw4CYGL4s0IcPgI8PcP48YGNj6Mi0Qu2u2OPHj4MxhtatWyMyMhKlSpWSH7OwsICLiwvKmmhF55xQVywhhZvRFHBNTQWaNuW7kgBg0CBg0ya9hpBbweZbL28h+GQwwq6HIZNlys8pal4UPzb4EeObjodzMQONSyTkS1/Xfxw0CNi40WjrP+q03MmjR49Qvnz5QjP9nBI7QohR+OUXvgsJ4FvpLlzgS5zoSW4FmxuPbYxzxc+BIet4ccviGNNoDPya+MHexl5vsRKiltu3+RVbPnwe97lhA2Ckk0B1mtidOHEix+MtW7bU5HJGjxI7QojBRUUBPXvy31tZ8UWIa9fW2+1zK9gMACgOwB+AGVDaujTGNhmLnxv9DDsrO/0ESUhehIUB/frx31tZ8RORvv3WsDEpodM6dsqKcX7ZeieVSjW9JCGEEFXi4vgxQTIhIXpN6gAgJiYm56QOAJKBki9KYvrA6RhZfyRsLfTXmkhInvn68uPtVq/mhzt8/z1fCsWEG3I0rmP39u1bha8XL17gwIEDaNiwIQ4dOqSLGAkhpHDKyOBLMbx7x2/36gWMHKn3MNQtxLyk+RKMazqOkjpiWpYsAerV47+/d4+fKatZZ6ZR0bjFrkSJEtn2tWvXDpaWlhg7diwuXbqklcAIIaTQCwjgu10BwM0NWLtW74O7M1km7qXfU+tcFyHNdCUmyMoK2LWLT+6Skvjv3d0VV3YxIRondqo4ODjgzp072rocIYQUbv/8AyxcyH9vbs7Xq1Pyh7WuSDIl2HlzJ4JignDr+S1+DF2y8nNlBZvd3d31Fh8hWuXmxs8y79EDUgAxY8ci8d07OLu7G24mfB5pnNhdu3ZNYZsxhsTERMybNw/fGuGAQ0IIMTnPnvHlF2QWLAAaNNDLrdOl6dh6dSvmnZqH+2/u8zvNAHQEsBMAB3wx8dXwBZsJ0Zbu3RHVrRv89uzBE6kUmDkTgGJJH1Og8axYMzOzbJXYAaBJkyb4448/ULUAVW8GaFYsIUTPMjOBTp0A2ZjlLl2APXt03gX7KeMT/rj8B+afmo/45HiFY81EzTCj5Qx8uPoB/v7+ChMpRCKR4Qo2E6JFuZX0iYiIMNjnXOd17L5kZmYGBwcHhSXGChJK7AgherV4MTB+PP+9szNw7Rpgr7sacO/T32P1xdVYdGYRniU/Ax4BeA/AFmjdqjVmtJoBDxcP+S83oynYTIgW5VbSx9DrQ+u03ImLCw2OJYQQnbh8GZgyJWt7yxadJXXvUt9h+fnlWHJ2Cd58egPcAnAACuPo7h69izehb8C5ZrUWCgQCpWWvCDFluZX0YYwhPj4eMTExRv/517jcCcAvL9a1a1dUrFgRlSpVQrdu3RATE6Pt2AghpPD48AHo25cvcQIAEycCbdtq/TYvP7xEwNEAuIS4YIZ4RlZStxPZJkckJCTAx8cHUVFRWo+DEGOibkkfdc8zJI0Tu23btqFt27awsbHBmDFjMHr0aFhbW6NNmzYICwvTRYyEEFLw+fsDssoC9esDgYFavXxiSiLGHxwP11BXzD05F8lpfBZnxsxgc1T54ueykTr+/v5UfJ6YDKlUiujoaISHhyM6Olqtz66zs3prGDs7OuY3PJ3TeIxdtWrVMHLkSIwdO1Zh/+LFi7Fu3Trcvn1bqwEaGo2xI4ToXEQEX/EeAIoWBf79F6hcWSuXfvTuERacWoANlzcgTZom329uZo7BdQbDg3mgv1f/XK8jFouNvguKkKioKPj5+Sl0q6ozq1U2xi4hISHb5AmAnwwuBBAbEACBlv/oUocmuYjGLXYPHz5E165ds+3v1q0bYmNjNb0cIYQUbvHxwIgRWdvLlmklqbv3+h6G/jUUFZdVxMqLK+VJnVURK/zS6Bc8GPMAa7uuhdkH9X4NmEIXFCncZLNavx4rp86QAoFAgNDQUACKy6QCfFIHACEABHPnAgcPajFq7dM4sROJRDh69Gi2/UePHoVIJNJKUIQQUihIpUD//opLhg0enK9L3nhxA76Rvqi6oio2XtkISaYEAGBrYYtJzSYhzi8OSzsthagE//+12l1Qap5HiCFIpVL4+fkpbW1Td0iBt7c3IiIiUK5cOYX9QpEIEf37w5u/GNCvH/8HmZHSeFbs+PHjMWbMGFy5cgXNmjUDx3E4efIkNm3aJM92CSGEqCE4GDhxgv++fHlgzZo816u7+PQigmKCsPu/3Qr77azsMKbRGIxpPAalbUpne5y7uzuEQqHqLihaVYKYAG3NavX29oaXl1f2kj4cx/8Btm8f8Po10Ls3cPw4vyqMkdE4sfvpp59QpkwZLFq0CDt37gTAj7vbsWMHvLy8tB4gIYQUSGfOALNn89+bmQF//gnY2Wl8mVOPTyEwJhAH7h9Q2G9vY4/xTcdjVMNRKG6pekyOrAvKx8cnW/F5WlWCmAptzmpVWdJn82Z+PdlHj/if3ylTgEWLNIxU9/K0VmyPHj3Qo0cPbcdCCCFqMfkiuUlJgK8v3xULADNmAC1aqP1wxhiOxh5FUEwQouOiFY6VLVYWE5tNxIh6I1DUoqjCMVWvm6wLStmgc1pVgpgCvQwpKFUK2LmT/1nNyOCLibdoARhZPqTxrNjChmbFEmJc8jrrzWjIxuiEh/PbzZsD0dFAkdz/zmaMYf+9/Qg8EYhzCecUjrnauWJK8ykYXGcwLItYZnusOq+bySfMpNDKdVarNleOWL4c+OUX/vsSJYBLl4BvvsnfNXOh0yXFChtK7AgxHsa8lqPatmwBBg3ivy9RArh6FchlRR9pphRRt6MQFBOEq8+vKhyrUroKpraYCt9avjAXKB/vUyBeN0JyIfucA1A6pEBrn3PG+DF2u3bx23XrAqdOAdbW+b+2CpTYaREldoQYB2Nfy1Et9+/zvwTev+e3w8OBPn1Uni7JlCD8ejjmnpyL/179p3CslmMtTG85HT2r9YTATPXzLRCvGyFqUtYyLRKJtD+kIDkZaNAAuHeP3x4yBNiwIc+Tn3K/HSV2WkOJHSHGITo6Gp6enrmeZ7SFdDMy+G7XCxf47cGDgY0blZ6aJknD5qubMf/UfDx8+1DhWMOyDTG95XR0qdwFZlzuFatM/nUjREN6G1Jw/TrQuDHw6RO/vXatYk1KLdIkF8nT5AlCCNE3k1/LcebMrKSuUiW+EPFXPmZ8xPp/12PBqQVISElQONbSpSWmu09HW7e22Qqo5sTkXzdCNKRyVqu21aoFrFvH16IE+Ba7YcP4We4GpHFiJ5VKsWnTJhw9ehQvXrxAZmamwvFjx45pLThCCJEx6UK6x44B8+fz3xcpAoSFAba28sMpaSlYeWElFp9djBcfXig8tP037RHgHoCWLi3zdGuTft0IMXb9+gHnzvFdsytXGjypA/KQ2Pn5+WHTpk347rvvULNmTY3+ciSEkLwy2UK6r18DAwbwA64BICiIH5sD4O2nt1h6bilCz4XibepbhYd5VfFCgHsAGpZrmK/bm+zrRoipWLKET+iMJB/SOLHbvn07du7cic6dO+siHkIIUcokC+kyxnfNPH3Kb7dpA0yYgBcfXmDJmSVYcWEFUtJT5Kdz4NCrRi9Mc5+G2k61tRKCSb5uhJgSI/vZ0bjN0MLCAhUrVtRFLIQQkiOVazkKhcZZsmPNGuCvv/jvS5dG4or58D80Dq4hrph3ap48qRNwAgz6dhBu/3wb2322ay2pkzG5140Qkmcaz4pdtGgRHj58iOXLlxeKbliaFUuI8TGJQrq3bgH16wOpqQCAFTM6YJyFGOnSdPkpFgILDK0zFJOaT0KFkhV0HpJJvG6EkGx0Wu6kR48eEIvFKFWqFGrUqAHzrxbAjYqK0jxiI0aJHSFEY2lpfBmEq3wx4ZUNOfz8XdZ/tdZFrPFD/R8wodkElCteTtVVCCEEgI7LndjZ2dE6sYQQo2NMrVEv/YbD4XNSd8MBGN+eT+qKWRTD6Eaj4d/EH45FHQ0SGyGkYNM4sduooqAmIYQYirGsH3s+4Tz+XuaH2WvOAgBSBYBvT8CmeClMbeyHXxr9gpLWJfUWDyGk8MlzgeKXL1/izp074DgOlStXhoODgzbjIoQQtahaBzUhIQE+Pj56mRxw4tEJBJ4IxJVrh3FtVdb+wO9sMaD/TPzY4EcUsyym0xgIIQTIwxi7Dx8+4JdffsGWLVvkxYkFAgEGDhyIZcuWwcbGRieBGgqNsSPEeBlyHVTGGA49OISgmCDEPI4BGLAnHOh6lz/+qEk1OB6/CGuLgvV/IiEFnTEN65DR6Ri7cePG4fjx49i7dy+aN28OADh58iTGjBmD8ePHY9WqVblcgRBCtCMmJkZlUgfwyVd8fDyio6MhEAi08h91JsvE3jt7ERgTiItPL8r3j7qQldQxBwe47BYDlNQRYlKMZVhHfmjcYmdvb4+IiIhs67CJxWL06tULL1++1GZ8BkctdoQYr/DwcPj6+uZ6XqlSpfDmzRv5dl7+o5ZmSrHr1i7MjZmL6y+uKxzrklEB//v9CYqkZfA79u8HqIg7ISZF1bAOWWk3Q9Z81CQX0bhA8cePH+Hk5JRtv6OjIz5+/Kjp5QghRGNSqRTR0dG4deuWWud/mdQBWePv1CnPlCHNwKYrm1B9ZXX0jeyrkNTVKVMHkd3+xJ49tllJ3S+/UFJHiImRSqXw8/NTuuyebJ+/vz+kUqm+Q9OYxold06ZNMWvWLKR+LroJAJ8+fcKcOXPQtGlTrQanzMqVK1GhQgVYWVmhfv36iImJyfH848ePo379+rCysoKbmxtWr16t8xgJIboTFRUFV1dXeHp6IjAwME/XUOc/6lRJKlZdWIVKyyphyF9DcPf1XfmxxuUaY1/fffh35L/w3nQO3PXPyV7NmsCCBXmKiRCiX7I/EMPDw7Fs2TK1hnXklnMYA43H2IWGhqJjx44QCoX49ttvwXEcrly5AisrKxw8eFAXMcrt2LED/v7+WLlyJZo3b441a9agU6dOuHXrFsqXL5/t/NjYWHTu3BkjRozAtm3bcOrUKYwaNQoODg7o2bOnTmMlhGifqq6SvPjyP+ovh5Z8SP+AtZfWYuHphUh8n6jwGE9XTwS4B6B1hdZ898w//wBLl/IHLS2B8HDAyirfsRFCdEvZWDp1JCYm5n6SgWk8xg7gW+i2bduG//77D4wxVK9eHf369YO1tbUuYpRr3Lgx6tWrpzBBo1q1aujevTuCg4OznT958mTs2bMHt2/flu/78ccfcfXqVZw5c0ate9IYO0KMQ24zYL/29bg6VcLCwtC3b18kpyVjxfkVWHx2MV59fKVwTqeKnRDgHoDm5Ztn7Xz+HKhdG3jxgt9eupTvhjVCxjjLjxBDyc8fiEuWLIGTk5Pef450OisWAKytrTFixIg8BZdX6enpuHTpEqZMmaKwv3379jh9+rTSx5w5cwbt27dX2NehQwds2LABGRkZ2ZZDI4QYr9xmwMpMnz4dbdq0gVQqRdu2bXM936akDWaKZ2LZ+WV4l/pO4ViPqj0Q4B6A+mXrKz6IMWDo0KykrnNnYPRodZ+KXhWEWX6EaEtOY+lyIxAIMHbsWPm2sf4cqZXY7dmzB506dYK5uTn27NmT47ndunXTSmBfe/XqFaRSabaJG05OTnj27JnSxzx79kzp+RKJBK9evYKzs3O2x6SlpSEtLU2+nZycrIXoCSH5pW4XSPXq1dGqVStIpVIIhUIkJCQo/U+c4zjY2tvC96IvPkqzJn6ZcWboU7MPpraYipqONZXfZPly4O+/+e+dnICNG4HPM+eMiTEUbybEmKj7B6IyX4/HNdafI7USu+7du+PZs2dwdHRE9+7dVZ7HcZzOZ4xwX/3nyRjLti+385XtlwkODsacOXPyGaXxo64ZYmqU/SGW03kCgQChoaHw8fEBx3HZkhvGGFI8U4DP/2UVMSuCgbUHYkqLKahUupLqG1y/DkycmLW9aRPgaHzrvuY2y4/jOPj7+8PLy4t+9kmhkZcxcgKBQGluY6w/R2rNis3MzITj5/+4MjMzVX7pMqmzt7eHQCDI1jr34sULpeVXAKBMmTJKzy9SpAhKly6t9DFTp05FUlKS/Cs+Pl47T8CIfDmr0NfXF56ennB1dVWr9AMhhuLu7g6hUKjyjzKO4yASieDu7i7f5+3tjYiICJQrV07x5OIAegGoDlgKLPFzw59x/5f72OC1Ieek7tMnoG9fQNaq7+cHdOyYvyemI+oWbzaFWX6EaIu6fyAuWbIEYWFhWLJkSY65jTH+HGlc7kSZd+/eaeMyObKwsED9+vVx+PBhhf2HDx9Gs2bNlD6madOm2c4/dOgQGjRooHJ8naWlJYoXL67wVZDIuma+/g9fk7pehBiCrAUOyN7iLtsOCQnJ9ldzVfeqaLmkJbjBHNATwCAA/oDNtzYY33Q8Yv1isbzzcrjYueQexKRJwM2b/Pe1awPz5uXzWemOui0TpjDLjxBtUfcPxF9++QV9+/ZV2XD0NaP6OWIamjdvHtu+fbt828fHh3Ecx8qWLcuuXLmi6eU0sn37dmZubs42bNjAbt26xfz9/VnRokVZXFwcY4yxKVOmsAEDBsjPf/jwIbOxsWFjx45lt27dYhs2bGDm5uYsIiJC7XsmJSUxACwpKUnrz0ffJBIJEwqFDIDSL47jmEgkYhKJxNChEqJSZGRkts+xSCRikZGRCuf9+/Rf1nNHT8bN5hhmQ/5VPLg4CzgawF5+eJnrvSQSCROLxSwsLIyJ585lEn7aBGNWVozduKGrp6gVYrFY5c/6l19isdjQoRKiV5GRkYzjOMZxXLbfgRzHKfxfYiw/R5rkIhondhUqVGCnTp1ijDF26NAhZmdnxw4ePMiGDRvG2rVrp3m0GlqxYgVzcXFhFhYWrF69euz48ePyY4MGDWIeHh4K50dHR7O6desyCwsL5urqylatWqXR/QpSYmcsH1BC8ksh4RKLFf4YOf34NPvuz+8UkjnMBis9vzQLPB7I3n56q9Y9lCWQQoBFAoytWKGjZ6Y9sj/kvv7lRX/IEaL+H4jG8nOkSS6icR07a2tr3L17FyKRCH5+fkhNTcWaNWtw9+5dNG7cGG/fvtXkckavINWxU3ddTVldL0JMBWMM0XHRCIwJxLHYYwrHytiWwYSmE/BDgx9ga2Gr1vVUrhn5+d+IiAh4m0CRc9nzAKDwXIxh7UtCDE3dSYTG8HOk07ViS5YsKZ9QcODAAXmdKMaYSayhVphpOquQEGPHGMPf9/5Gi40t0HpLa4WkrnyJ8ljReQVi/WIxvtl4tZO6HGeTAgDHwX/sWJP4/07V5BGhUEhJHSn0BAIBWrVqhb59+6JVq1YqZ7Wa2s+Rxi12o0ePxr59+1CpUiVcvnwZcXFxsLW1xY4dOzB//nz8+++/uorVIApSi52scn9Odb2EQiFiY2ONZto2Icpkskzs/m83Ak8E4vKzywrHKpaqiKktpqJ/7f6wEFhofO3o6Gh4enrmep5YLFZYisxQ1Gl1oPJGhOSfIX+OdLryxJIlS+Dq6or4+HgsWLAAtrb8X8GJiYkYNWpU3iImepFTXa+cZhUSYiwkmRLsvLkTQTFBuPXylsKxGg41EOAegO9rfI8iZnlaVAeAac0mVXdVCVnLBCEk70zl5yhPa8UWJgWpxU5G2S8DkUiEkJAQo2tSJgQA0qXp2Hp1K+admof7b+4rHKvvXB8B7gHwquoFMy7/FZxMpcVO5ThAGj9HSIGjSS6icWK3efNm2Nvb47vvvgMATJo0CWvXrkX16tURHh4OFxc1akGZkIKY2AHUNUNMw6eMT/jj8h+Yf2o+4pMVi4U3EzXDjJYz0OGbDjmuPqMpqVQKVycnJLx+DWX/ORrDkAXZsApVBYiNIUZCiPbodPLE3LlzYW1tDQA4c+YMli9fjgULFsDe3l5hcVxi3NQdNEqIIbxPf4/fT/8Ot6VuGP3PaIWkrk2FNhAPEuPkkJPoWLGjVpM6ABAkJCD00ycAWbNgZYxlyAKtKkEIUUXjgSjx8fGoWLEiAGD37t3w8fHByJEj0bx5c5PoeyaEGK93qe+w/PxyLDm7BG8+vVE41qVyFwS4B6CJsInuApBIAF9feH/8iAgAftbWePI5yQP48WvGMGTBlMYBEkL0S+PEztbWFq9fv0b58uVx6NAheSudlZUVPn3xHyAhhKjr5YeXCDkbguUXliM5LVm+nwOHntV7YlqLaajrXFf3gfz6K3DqFADA28UFXpcuIeb6daMbskCliwghqmic2LVr1w7Dhw9H3bp1cffuXflYu5s3b8LV1VXb8RFCCrDElET8fvp3rL60Gh8zPsr3CzgBfGv5YmqLqajmUE0/wURHA4GBnwMQAOHhEJQubZQ9EbL1LnMrXeTu7m6A6AghhqRxYrdixQpMnz4d8fHxiIyMROnSpQEAly5dotUK8oEmM5DC5NG7R1hwagE2XN6ANGmafL+5mTkG1xmMyc0n45tS3+gvoFevgH79+JVgAb7lrmlT/d1fQ1S6iBCiCpU7yYU+ZsWqW4uqIDKWhNZY4ijo7r2+h+CTwdh6bSskmRL5fqsiVhhRbwQmNpsIUQmRfoNiDPDyAvbu5bdbtwYOHeJb7YwclS4ipHDQabkTgJ+RtWbNGjx8+BC7du1CuXLlsHXrVlSoUAEtWrTIc+DGSNeJnaFrURkyoTGWhNZY4ijIbry4gbkxc7Hj5g5kskz5flsLW4xqMArjmo6Dk62Tzu6f4+d86VLAz4//3t4euHoVKFtWZ7FoG/1RQkjBp1EuwjQUERHBrK2t2fDhw5mlpSV78OABY4yxFStWsE6dOml6OaOXlJTEALCkpCStX1sikTChUMjAL0GZ7YvjOCYSiZhEItH6vRljLDIyMtv9hUIhi4yM1Mn9vr43x3FKnzPHcXqJwZjiKKguJFxg3bd3Z5gNhS+7eXZs5rGZ7NWHVzqPIcfP+b//MmZhwRjfbsfY/v06jycnEomEicViFhYWxsRisc5+9gkhpkWTXETjxK5OnTps8+bNjDHGbG1t5Ynd5cuXmZOTk6aXM3q6TOzEYrHKpO7LL7FYrPV7GzKhMXRCa2xxFEQnH51kHbd1zJbQ2S+wZ8ExwSwpVfs/T8rk+jl3ds5K6saO1UtMOcVqqD+0CCHGTZNcROMCxXfu3EHLli2z7S9evDjevXun6eUKNUPVopJKpfDz81M6m062z9/fH1KpVKv3lTGW4qrGEkdBwRjDkYdH4LnZEy02tsCB+wfkx8oWK4slHZYgzi8OU1pMQXFL3a/ikuvnnDH4JyZCCgD16gHBwTqPSRXZkIyvP48JCQnw8fFBVFSUgSIjhJgajRM7Z2dn3L9/P9v+kydPws3NTStBFRbOZcqod56Wa1EZOqExluKqxhKHqWOMYd/dfWi6oSnabW2H6Lho+TFXO1es/m41Ho55CP8m/ihqUVRvceX6OQcQDyDGygrYvh2wtNRbbF8y9B9ahJCCReNyJz/88AP8/Pzwxx9/gOM4PH36FGfOnMGECRMwc+ZMXcRYYLm7u0NobY2ET59yXJNS27WoDJ3QGEtxVWOJw1RJM6WIuh2FoJggXH1+VeFYldJVMLXFVPjW8oW5wNwg8an9OR8yBKhUScfRqKbJH1rGWFOPEGJcNE7sJk2ahKSkJHh6eiI1NRUtW7aEpaUlJkyYgNGjR+sixgJLUKQIQjdsgI+vLzhAIbnTZS0qQyc0xlJc1VjiMDWSTAnCr4dj7sm5+O/VfwrHajnWwvSW09GzWk8IzAw7M1Ptz3mvXjqOJGeG/kOLEFKwaNQVK5VKcfz4cYwfPx6vXr3C+fPncfbsWbx8+RK//fabrmIs0Lz79kXE+vUo91XyJhQKdVbqRJbQqFo8neM4iEQinSU0suKqsnt9fW9AP8VVjSUOU5EmScPaS2tRZXkVDNw9UCGpa1i2If7q8xeu/HgFvWr0MnhSB6jxOQcgKlfO4Im7of/QIoQUMJrOzLC0tGQPHz7U9GEmS5ezYr8kuXCBiS0tWRjAxACT/PqrTu8nmy349YxBfZb5UDYLUCQS6X0WoLHEYaw+pH9goWdDWblF5bLNcm25sSU7eP8gy8zMNHSYSqn8nH/+rBvDeyybna1s9q7sZ1LZ7GwqjUJI4aHTcicNGjRgR44cyVNgpkhfiR1jjLH//Y8xjssqv7Bzp/yQLv4TN4aExlh+ORlLHMYkOTWZzYuZxxwXOmZL6Npvbc+Oxx03dIhqiYyMZMIyZRQ/5yVLGkVSJ6PpH1pUGoWQwkWnid3BgwdZnTp12N69e9nTp09ZUlKSwldBo9fEjjHG5s3LSuysrRm7cEGn/4lTQkO+9ubjGzZbPJuVnFcyW0LnFe7Fzj85b+gQNfPpE5PUqcPEAN8i3qYNk2RkGDqqbNT9Q4uKahNS+GiSi2i8pJiZWdawvC/HrjDGwHFcgZuSr4+1YhUwBgwZAmzeDACIsrODT1KSwZYcI4XHiw8vsOTMEqy4sAIp6Sny/Rw49KrRC9Pcp6G2U20DRphHo0YBq1bx31euDFy8CBQrZtiYVMhteTCpVApXV1eVs2hlE35iY2NpbCghBYhO14o9fvx4jsc9PDw0uZzR03tiBwBpaUC7dpDGxMAVgKpCCPSfONGGhOQELDy9EGsvrcUnySf5fgEnQP/a/TG1xVRUsa9iwAjzYccOoE8f/nsrK+DcOaC2CSann0VHR8PT0zPX88RiMZVGIaQA0SQX0bjcSUFL3IySpSUQFYWYWrXw5NkzlacxI6hvRQuQm67Yt7GYf2o+Nl7ZiHRpuny/hcACQ+sMxaTmk1ChZAWd3V/nn51794ARI7K2ly0z6aQOoNIohJDcaZzYXbt2Tel+juNgZWWF8uXLw9JAFdwLFHt7JI4bB0yalOuphvpPPCoqCn5+fgrdQkKhEKGhodQ9bMT+e/Ufgk8G489rf0LKsoZOWBexxg/1f8CEZhNQrng5ncag889Oairw/fdAyucu5X79gGHD8n9dA6PSKISQ3ORpjJ2qulAAYG5ujt69e2PNmjWwsrLKd4CGZpCu2M+MudtFtrYljf0zHdeeX0NQTBB23dwF9kU57GIWxTC60Wj4N/GHY1FHncehl8/Ojz8Ca9bw31epwo+rs7XN3zWNgGyMXW5FtWl4BiEFi07H2P3111+YPHkyJk6ciEaNGoExhgsXLmDRokWYNWsWJBIJpkyZgt69e+P333/P1xMxBoZM7OT/iT95kuOSY/r+T5wGcJuW8wnnERQThD139ijsL2VdCn6N/fBLo19Q0rqkXmLRy2cnPBzw9eW/t7bmx9XVqpXHiI2PLDEGoJDc0R9VhBRcGuUimk65bdiwITtw4EC2/QcOHGANGzZkjDH2v//9j7m5uWl6aaOk93InX5HXtzKi0gZisVhpIdWvv8Risd5jI1mOxx1n7ba0y1ayxHGhI1twcgFLTk3We0w6/+zcusVY0aJZJYM2bNBq/MbCGGpQEkL0R5NcROMxdtevX4eLi0u2/S4uLrh+/ToAoE6dOjR4V0u8vb0REREBvzFj8CQhQb5fKBAg5I8/DPKXOQ3gNl6MMRx6cAhBMUGIeRyjcExYXIhJzSZheL3hsDa3Nkh8Ov3spKQA3t7Ahw/89oABfOmgAsjb2xteXl40cYkQko3GiV3VqlUxb948rF27FhYWFgCAjIwMzJs3D1WrVgUAJCQkwMnJSbuRFmJeXl4oUaIEog8cADZuRKvXr9FKIoEgLAzo2xcoovHbmC80gNv4ZLJM7L2zF4Exgbj49KLCMbeSbpjSfAoGfjsQlkUMO7FJZ58dxoChQ4H/Pq9fW7s2sHo1kMN4YFMnEAiopAkhJBuNx9idPn0a3bp1g5mZGWrXrg2O43Dt2jVIpVLs27cPTZo0wdatW/Hs2TNMnDhRV3HrjSHH2AEqZg+amSE0MxPeAPDLL8DSpXqNiQZwGw9pphS7bu3C3Ji5uP7iusKxqvZVEeAegD41+6CImX6Tf1V09tlZtAiYMIH/vkQJfrJExYpaipoQQgxLp5MnAOD9+/fYtm0b7t69C8YYqlatCl9fXxQz0mru+WHIxE7l7MHP/0YAfHK3fDnw888GiQ2gAdyGkCHNwJ/X/0TwyWDcfX1X4VidMnUQ4B4A72reMOPMVFzBcLT+2Tl+HGjTBpCtevPXX0C3blqLlxBCDE3niV1hYqjELtfZgwCEAGLBd8lg/36gQwe9xQcob00UiUQICQmhpC6PcivamypJxcbLGzH/1Hw8Snqk8NjG5RpjRssZ6Fypc44liYyB1j47T58C9eoBz5/z2wEBQGCglqMlhBDD0nlit3XrVqxZswYPHz7EmTNn4OLigiVLlsDNzQ1eXl55DtwYGSqxU7uGHYBWAFC8OHDmDFC9uo4jU0QrT2hPTkV7O3TpgLWX1mLh6YVIfK84scDT1RMB7gFoXaG10Sd0X8r3ZycjA/D0BE6d4rfbtQP++Qegzx8hpIDR6ZJiq1atwsyZM+Hv74/AwEBIP3d/lCxZEiEhIQUusTMUtWcPNmjAjydKTga6dOFrdjk46CQmVb+IaQB3/qnqdk9ISEDPnj1RbEAxpHyTonCsU8VOCHAPQPPyzfUZqtbk+7MzcWJWUle+PBAWRkkdIaTQ03gAzrJly7Bu3ToEBASgyBezMRs0aCAvd0LyT+3Zg7/+CtSty2/ExvLlHtLStB5PVFQUXF1d4enpCV9fX3h6esLV1RVRUVFav1dhI5VK4efnp3QygWxfyl8pQCa/r0fVHrg44iL+7ve3ySZ1+RYeDoSG8t9bWAAREYC9vWFjIoQQI6BxYhcbG4u6skTiC5aWlvggqx9F8s3d3R1CoVBl1xrHcRCJRHBv3x7YsweQJYInTwIjR/LlH7RE1pr09Xi/hIQE+Pj4UHKXTzExMSrHUsolA20EbXD9p+uI6h2F+mXr6ye4fJBKpYiOjkZ4eDiio6Plrfv5duMGMHx41vayZUDDhtq5NiGEmDiNE7sKFSrgypUr2fb/888/qK7n8V0FmUAgQOjnFomvkzvZdkhICD8mSSjkkzvrz0Vnt2wB5s3TShzqtCb5+/tr75d2IaRut/uwSsNQ07GmjqPRDp218L55A3h5AR8/8ttDhgAjRuQ/YEIIKSA0TuwmTpyIn3/+GTt27ABjDOfPn0dQUBCmTZtWIOrWGRPZqhPlypVT2C8UCrOXhGjQANi6NWt72jQgMjLfMeTWmsQYQ3x8PGJiYlSeQ1R78OYBtj3cpta5plLwWWctvBIJ0Ls38PAhv12/PrBiRYEuQkwIIZrK06zYdevWITAwEPHx8QCAcuXKYfbs2Rg2bJjWAzQ0QxcoBjScPTh3Ll/yAeBb8E6c4JO+PN5n586d8JUtqJ6DsLAw9O3bV92nVOjdenkLwSeDEXY9DJnSTCAEQLLyc02p4HOuZXry81zGjQOWLOG/d3TkJw2JRPmMmBBCjJ9OZ8UCwIgRIzBixAi8evUKmZmZcHR0BMD/Rf516xLJP41mD06dyi+rtHUr8OkT3211/jyQy/uiqtTGCDW7uUylNcnQLideRlBMEKJuR4Hh899UZoB1V2t8+vMTOI5TWrRX3u1u5DRp4dVoRuzmzVlJnbk53xpNSR0hhGSTr7L09vb2cHR0xLNnz/DLL7+gIi3hY3gcB6xbBzT/PFvy6VOga9eshdGVyKnrbNasWShdunTukzjc3bX2FAqiM/Fn0CWsC+qtrYfI25HypK60dWkEegbi6fqniIyMVK/b3YipXaZHzfMA8H+Y/PBD1vby5UCLFhpGRgghhYPaid27d+/Qr18/ODg4oGzZsli6dCkyMzMxc+ZMuLm54ezZs/jjjz90GStRl6Ul8L//ARUq8NuXLwP9+wOZmdlOzW1yxJcJXa6TOIgCxhjEsWK02dIGzf5ohv339suPlbEtg9/b/Y44/zgEtAyAnZUdvL29ERcXB7FYjLCwMIjFYsTGxppMUgdoUKZH3RbexESgR4+sEj4//cTP+iaEEKKU2mPsRo0ahb1796J37944cOAAbt++jQ4dOiA1NRWzZs2Ch4eHrmM1CGMYY5dnt24BTZvyxYsBYNIkYP58hVPUXeFizpw5WLduHS0fpgbGGP65/w+CYoJwOv60wrHyJcpjcvPJGFp3KKyKWBkoQt2RjbFLSEhQ+seCRmPs0tKAVq2As2f57ZYtgcOH+bp1hBBSiOhkjN3+/fuxceNGtG3bFqNGjULFihVRuXJlhISE5DdeoivVqwM7dwKdO/OtdQsWAN98o9DioW6XWKVKlRAXF0fLh+Ugk2Vi93+7EXgiEJefXVY4VrFURUxtMRX9a/eHhaDgJiayMj0+Pj75Gy/IGN86J0vqRCJg1y5K6gghJBdqJ3ZPnz6V16lzc3ODlZUVhn9ZJJQYpw4dgKVLgdGj+e1Ro/i6d507A9Cs64yWD1NOkinBzps7ERQThFsvbykcq+FQAwHuAfi+xvcoYpanuUomR1amR9lkHLVbeBcuBDZu5L+3tgZ27+ZnwhJCCMmR2l2xAoEAz549g8PndUiLFSuGa9euoYJsHFcBZdJdsV+aMAFYtIj/vmhR4PhxoH597XadFTLp0nRsvboV807Nw/039xWO1XeujwD3AHhV9YIZl685SiZLozI9X/rf/4CePbNWT9m+na9fRwghhZROumIZYxg8eDAsLS0BAKmpqfjxxx9RtGhRhfNoeSndyPMvSZkFC4DHj/nurA8fgC5dgDNnIHB11U7XWSHyKeMT/rj8B+afmo/45HiFY81EzTCj5Qx0+KaDypnEhUWeWngvXQL69ctK6n79lZI6QgjRgNotdkOGDFHrghtl3ScFhDG02KmqMRcaGqrZxIXUVKBtW+DUKX67WjX++5Illd6DJkcoep/+HqsvrsaiM4vw7P0zhWNtKrTB9JbT4eHiUegTujxLSAAaNeJL9AD8TO4tW2hlCUJIoadJLpKnlScM4e3btxgzZgz27NkDAOjWrRuWLVsGOzs7lY8ZPHgwNm/erLCvcePGOCsbkK0GQyd2shpzX79NsuRB4xpnr18DzZoBd+/y2y1bAocOAZaW+W8VLKDepb7D8vPLseTsErz59EbhWJfKXRDgHoAmwiYGiq6AeP+e/yxe/jzppHlz4OhRvnQPIYQUcgUysevUqROePHmCtWvXAgBGjhwJV1dX7N27V+VjBg8ejOfPnyu0IlpYWKBUqVJq39eQiZ3Olmd6+JAvg/LiBb/dpw/w55+AWeEcC6bKyw8vEXI2BMsvLEdyWtZ6Xxw49KzeE9NaTENd57oGjLCAkEr5MXV//cVvV6gAnDsHfB7PSwghhZ3OlxTTt9u3b+PAgQM4e/YsGjduDIBfr7Zp06a4c+cOqlSpovKxlpaWKFOmjL5C1SqdLc/k5gbs2wd4ePDLjm3fDri4APPm5T/oPDC2lsLElET8fvp3rL60Gh8zPsr3CzgBfGv5YmqLqajmUM1g8RU4kydnJXXFi/OfTUrqCCEkT0yiiebMmTMoUaKEPKkDgCZNmqBEiRI4ffp0Do/kC/A6OjqicuXKGDFiBF7IWqlMgE6WZ5Jp2BDYsSOrlW7+fGDVKs2vk09RUVFwdXWFp6cnfH194enpCVdXV4NMwnn07hF+3v8zKoRWwOKzi+VJnbmZOUbUG4E7o+9gS48tlNRpU2ho1mxtgQCIiODrLxJCCMkTk2ixe/bsGRyV1LCSrVOrSqdOnfD999/DxcUFsbGxmDFjBlq3bo1Lly7JZ/d+LS0tDWmy5YvAN38aitaXZ/pa167AsmXAzz/z26NHA05OgJ4mS6gaP5iQkAAfHx+9rZF67/U9BJ8MxtZrWyHJlMj3WxWxwoh6IzCx2USIStCC81q3axcwdmzW9sqVQLt2houHEEIKAIO22M2ePRscx+X4dfHiRQDZ1ykFsq9l+rXevXvju+++Q82aNdG1a1f8888/uHv3Lvbv36/yMcHBwShRooT8SyQy3C90d3d3CIVClc+R4ziIRCK4u7vn/SajRvFLjQH86hR9+wLHjuX9emrKbY1aAPD394dUKtVZDDde3IBvpC+qrqiKjVc2ypM6WwtbTGo2CXF+cVjaaSkldbpw4gQ/61X2/s+YYfA1YKVSKaKjoxEeHo7o6GidfvYIIURXDNpiN3r0aPTp0yfHc1xdXXHt2jU8f/4827GXL1/CyclJ7fs5OzvDxcUF9+7dU3nO1KlTMW7cOPl2cnKywZI7rS3PlJt584Dnz4HNm4H0dMDLC4iOBurXz991c6Cz8YNquPj0IoJigrD7v90K++2s7DCm0RiMaTwGpW1Ka/We5As3b/KfsfR0fnvIEGDOHIOGpLWSQoQQYmAGTezs7e1hb2+f63lNmzZFUlISzp8/j0aNGgEAzp07h6SkJDRr1kzt+71+/Rrx8fE5dl1aWlqq7KY1BK0sz5QbjgPWr+dLoezbx5ee6NQJOHkSqFw5/9dXQqfjB1U4+fgkgmKCcOD+AYX99jb2GN90PEY1HIXilia8uogpePIE6NgRePeO3+7YEVizxqC16oxlSAAhhGiDSZU7efr0KdasWQOAL3fi4uKiUO6katWqCA4ORo8ePfD+/XvMnj0bPXv2hLOzM+Li4jBt2jQ8fvwYt2/fRrFixdS6r6Hr2MnoZebop09A+/Z8QgfwM2VPnQLKlcvT5XKKOTo6Gp6enrleQywW56vFjjGGo7FHEXgiEMcfHVc4VrZYWUxsNhEj6o1AUYuiKq5AtObdO8DdHbhxg9+uX59vGba1NVhIOispRAghWqRRLsJMxOvXr1m/fv1YsWLFWLFixVi/fv3Y27dvFc4BwDZu3MgYY+zjx4+sffv2zMHBgZmbm7Py5cuzQYMGscePH2t036SkJAaAJSUlaemZGLm3bxmrXZsxfvQTYzVqMPb6tcaXiYyMZEKhkAGQfwmFQhYZGckYY0wikTChUMg4jlM4R/bFcRwTiURMIpHk6WlkZmayvXf2ssbrGjPMhsKXa4grW31hNUvNSM3TtQsCiUTCxGIxCwsLY2KxOM+vs9o+fmTMwyPrc+XmxtizZ7q9pxrEYrHSz9/XX2Kx2NChEkIKMU1yEZOYFQsApUqVwrZt23I8h33R+GhtbY2DBw/qOiyjopVWPTs74MABoEULvpDxzZv8urKHDwNF1WvVUrdrSxfjB6WZUkTdjkJQTBCuPr+qcKxy6cqY1mIafGv5wlxgrtF1jUl+32e9jydLTwd8fIDjn1tM7e35z5gG42N1xRBDAgghRKd0nWWaOlNpscuthUxj9+8z5uSU1cLSti1jnz7l+jBZSxxUtHx83RKnLG6RSKRx3BnSDLblyhZWdXnVbC10tVbWYtuvb2cSqY5bpfQgv+9zZGSk0lZSjuMYx3F5/7x8QaE18MgRJvn++6zPUbFijJ0/n+97aAu12BFCTIEmuQgldrkwhcROZ7+sL19mrHjxrF/K333HWFpajg/Jyy/K/HQLpmaksjUX1zC3ULdsCV3DtQ3ZX//9xaSZ0rw9fyOT3/dZ06Q7rzFmSzwBFgkwZmXFWHR0nq+tC7oeEkAIIdpAiZ0WGXtip/Nf1qdOMVa0aFZy5+3NWEaGytPDwsLUSuzCwsJyfE65JXof0j+w0LOhrNyictkSupYbW7KD9w+yzMzMvD1nI6SN91nXrVMqE8/PX5HTpuXx2euWLO6vY9dmKyYhhOSHJrmISSwpRlTTpB5cnjRrxpdAsbLit6OigIED+YXblcjvahm5LTGWkpaC+Sfno0JoBfgd8ENCSoL8se2/aY/jg4/j+ODjaP9N+xyLV5sabbzPuhxPlmPB6c//+m/dapRFf2Ulhcp9NftbKBRSqRNCiMkxmckTRDm9DP5u1YpfpL1rV34gfHg4YGkJbNiQtdbsZ7LVMhISEpT+kpeVj1C2WkZuky6+n/09DlsextvUtwrHvap4IcA9AA3LNcz7czRy2nifdblEXVBQUM6JJ6CzgtPa4O3tDS8vL92XFCKEEB2jxM7E6Xw9WZn27fkF2r29AYkE2LQJsLYGVqxQKC6b19Uy1FlibOeinYA/ADOAA4deNXphmvs01Haqnb/nZgK08T7nJ+nOSVRUFGbNmqXWucY8u1QgEBhl0kkIIZqgrlgTp5f1ZGW6dgW2bwdkSdmqVcAvv/BrzH4hL11buXU1AgCSAbPHZhj07SDc/vk2tvtsLxRJHaCd91mWdMvO//rxgOYlZmQJubry/QcGIYSQHFFiZ+J08cs6Rz17Alu2ZLXSrVgB/Pij0uQuLi4OYrEYYWFhEIvFiI2NVTleSd2WnMXNF2NT902oYl8lX0/D1Gjrfdb2eDK1EvLPtPYHhh5IpVJER0cjPDwc0dHRRjk2kBBClNLdHI6Cwdhnxcpoqx6c2rZsYczMLGu27ODBjOWjJMTm3ZupnpgatPU+a2vlCXVnQQMwmdmlWq8JSQgh+aRJLmIya8UairGsFasOvawn+6Xt24H+/bNmyPr6Aps3A0XUH7p57fk1BMUEYef1nUAIgGTl56las1Pvz9kI5Pc5a/M1ix41Cp6rVuV63pw5czBz5sw83UOfVE3gkbWK0ixZQoghFMi1Yg3FVFrsDCYigrEiRRgDmARgYg8PFrZlS66tQOeenGPdwrsp1qDrpbpGm7J6YtSyojmtvWaZmYzNns0knwsQczm01AmFQpMo8KuPAs6EEJIXVKBYiyixU8OePSyySBEmVPIL/euE4XjccdZuS7tsRYUdFzqy+Sfns23bt6nV1aiPpbEKGq29ZpmZjI0fL++Gj5QVITbxAr+0vBghxFhRV6wWmVJXrKFERUXBp2dPfP1BknVf7dq1C7bf2iIoJggxjxUL6AqLCzGp2SQMrzcc1ubWAHLvKpRKpXB1dVU5aF9Vt21hprXXTCoFRo0C1q7N2rdkCaLKl4efn5/C9UUiEUJCQkym6zI8PBy+vr65nhcWFoa+ffvqISJCCOFpkotQHTuSL/L6c0qOyf5m6DuiLzJ+yVCYg+1W0g1Tmk/BwG8HwrKIpcLjcqsnpskqDKZWl0xXYwa18pqlpwNDhgBhYfw2x/EJ3vDh8AZMvsCv3mpCEkKIDlFiR/JFnXIXGW8zgEcAKgBV7asiwD0AfWr2QRGzvH389LLahgFERUVla/USCoUIDQ3Nd6tXvl+z5GTAxwc4fJjfLlIE2LoV6NNHfoqpF/jVVQFnQgjRJ6pjR/JF3YShSZIzdn2/CzdH3UT/2v3znNQBeW9ZMebaZLLZmF8nybLl1GRr5eZVvlqjnj4FPDyykjpLSyAyUiGpKwj0XhOSEEJ0Qaej/QoAmjyRM3UHnB+zsWHswAGt3FM2e1HZRAComL1ozDNo9TEbMy+vGWOMsZs3GStfPqteYalSjJ08mc9nbNz0XhOSEEJyoUkuQi12JM8+pH/ApSKXYFZC9ceIAyAC0PLjR+C774DVq/N9X01bVnTdGpZfmox/y6s8tUbFxADNmwOPH/Pbrq7AqVP8PhOiaUutpqumEEKIUdF9nmnaqMUuu3ef3rG5J+Yy+wX26tWfa9Qoq8UHYGzs2HytUiGjTsuKKdQmU3f1hrCwsHzfS+3WqE2bGLOwyHrP6tZlLDEx3/fXN2NuqSWEEHVRuRMtonInWV5/fI3Qc6FYem4pktKSFI41Sm6EuPA4vEh8Id8nL3fRvTsweTLw++9ZD+jWDfjzT8DWNl8x5TaLNDo6Gp6enrleRywWG2zgv75jzPE1k0r592rRoqwHdOgA7NoFFCuW73vrE60iQQgpKDTJRSixywUldsCz98+w+MxirLywEh8yPsj3m3Fm6FOzD6a2mIqajjVzL9Wxdi1fA03WFfbtt8D//gdUqKCz2E2hNpmsxlxuszF1XpcvKYmfEHHgQNa+n37C/9u797Ao6v0P4O8BBQQFRJCLrC6KpqZWqEfhSIqdNCoPSpBmJpbZwSQiszjqz4ISSEujsszKAzzlJeuA2sWSCowyU0yyvJQpBgfwqKmgmCDL/P6Ys6srILswu7OzvF/PM8/jzs7ufBgvffpePh+88grQubPl7msBrHVIRPaEdexIFuXV5Vj+7XK8s+8dXGq4ZDjfyaETZg6biX+O+Sf69+hvON9quYtHHpGSuJgYqXzGjz8Cw4cDGzZIo0IWoIbaZPr1bzExMRAEwSi5s9puzF9/BaKigMOHpdedOgGvvioldipkz7UOiYiuh5snqImjZ45iztY56PdqP6zas8qQ1Dk7OmPeyHn47bHfsDZqrVFSZ7Lbbwe++w7o/7/Pnj0LREYC6enSai6Z6WuTXbthQE8QBGg0GsVrk0VHR+PDDz9Er169jM4HBgZafspw0yYpwdYndV5ewPbtqk3qAPutdUhE1BqO2JHBwVMHkfFNBtb/tB6NYqPhvGtnV8wdMRdPhj4J/24yjGwNHgzs2QPMnAls3SoldIsXA8XFQFYW4OHR/nv8j02MhpkoOjraut0b6uqABQuAVauunLvxRun3pG9fy9zTStQwUktEZAlcY9eKjrDGbl/VPqQVpSH3UC7Eq5qDuTu747G/PIak0UnwdvWW/8aNjUBaGvDss1dG67RaaWp29GhZb9VcVwe19TKV1fHjwL33Sgm23owZUjkaNzfFwpKLzaxbJCKSATdPyMieE7vvyr9DWlEaPjnyidH5Hl164InRT2DeX+bB08XT8oF8+ilw//3AuXPSa0dH4Pnngaefln4tE0v1YVWdjRulaVb983Z2ltbTzZkj9X+1E/pdsQCaHanlrlgiUgsmdjKyt8ROFEUUHi/E0qKl+Kr0K6P3/Lr6YUHoAvxjxD/Q1cm0MiSyJUu//w5Mnw7s3Hnl3PjxUj/SgADzv4+aOnsWmDdPGhHV69tXKmUSEqJcXBbEkVoisgdM7GRkL4mdKIrY9ts2pBWlYWf5TqP3env0RvJfk/HQLQ/BpZOLyd8pe9P6hgYgNVWantX/sezeXSq3MWOGXY0mWd2XXwJxcUBFxZVz06YBq1cDnp6KhWUNHKklIrVjYicjtSd2jWIjNh/ejKVfL8W+E/uM3gv2CsbCMQsxY9gMODk6mfW9Fi3+WlgoTc1WVl45d9ddwJo1wDW7RqkV1dXAwoVSAqfn6Qm88QagUN0+IiIyDxM7Gak1sWtobMD7P7+P9G/ScfDUQaP3bvS5EYvDFyP2xlh0cjB/Y7RVir/+8QeQmAisX3/lnIcHsHIl8OCDHL0zRV4ekJBgnCDfdhuQnQ0EBioWFhERmcecXIR17OxMva4ea39Yi4GrBmJG3gyjpC7EPwS59+Zi/9z9uG/ofW1K6gDrNK1Hjx5Sy7HNmwE/P+lcdTUwezYwdiywf3/bv9velZUBU6YA0dFXkjo3N2lKe/t2JnVERHaMiZ2d+PPyn3h99+sIfjUYD3/0MI6ePWp4L0wThk+nf4riOcWYMmgKHIT2/bZbtfhrVBRw4IBU806vqEha7P/441d2dhJQWws88wxwww1SQqx3553SM0xMBBz4V56IyJ7xX3mVu1B/AS/tfAl9X+2LhG0JKK8pN7x3W9BtKIgrwDcPfoPI/pEtdl8wl9WLv3p5ATk5wLZtQHCwdE6nk0p0DBggFditr5fnXmrU2CjtHh4wQCoTc+l/7d98fYH33wc+/hjo00fZGImIyCq4xq4VtrrG7tylc1i1exVe3vUyzvx5xui9uwfcjcXhizE6UN4iv3qKFn+tqwNWrACWLgX+/PPK+b59pXNTp3acUSlRBLZskQo8Xz013amTNDq3ZInd73glIuoIuMbOjp2qPYXFXy5Gn8w+WFKwxJDUCRAQMzgGPzzyAz667yOLJXXAlTZdAJqMAlq8TZezM7BokdTX9N57r5w/dkyqgxcSIvU+1enkv7etEEWpqPPIkdJauquTukmTpGnXFSuY1BERdUBM7FSi6nwVnvz8SWhf0SL9m3TU1NUAABwFRzww7AEcePQAPoj9ALf432KVeBRtWg8AvXtL04x79kg7PfV+/FEatRs0CHjnHWmEz17U10tT0jffLJV/2bv3ynsjR0obI7ZulaZkiYioQ+JUbCuUnor9/dzvWP7tcqzdtxZ1uitJSmeHzph18ywk/zUZ/bz6WT0uPZsp/vrFF1K9tuJi4/N+flKrrDlzAI3G+nHJ4b//Bf71L+C114BrN6TcfDPw3HPA3XezBAwRkZ1iHTsZKZXYHfnjCDK+ycC7+99FQ2OD4bxLJxfMCZmDp8KegsZDXYmKxZNAUZQSvBdeAL4ybpcGBwdplOuRR4AJEwAn8woyW93ly9JmkX/9S9r8cO3U8qhRUi/dyZM7zppCIqIOiomdjKyd2P188mekF6Xj/QPvo1FsNJzv6tQVj454FPND58O3q6/F45Cb7O3HWvP998Dy5dLmgmuTou7dpRpvU6cCERHSZgNbUF8vdd3Iy5OO//7X+H1BkNbUPfkkEBamSIhERGR9TOxkZK3ErriyGGlFadh8eLPReU8XTyT+JRGJoxLRw7WHxe5vSRZtP9aaigpg7Vrg7beB5ooqe3kBf/sbcMcdwMSJQECAZeJoye+/AwUFQH4+8MknUhHmawUEALNmAQ89BPRTbtqdiIiUwcRORpZO7L4t+xZLi5bis98+Mzrv7eqNJ0OfxKMjH4W7s+2UWTGXVdqPmaKhAfjsM2DDBmkUr7a2+ev69gVGj5aO4cOlTRjdu8sTQ00NUFIC/PCDtPHhu++Ao0ebv9bFRZo6fughaerYVkYViYjI6pjYycjSid3E9yZi+9HthtcB3QLwVNhTmBMyB25ObrLfz9oKCwsRERHR6nUFBQUYN26c5QMCgIsXpXIhmzZJO0mbGyW7Ws+ewMCBUiuugADp8PICXF2lw9lZShwbGqS1cefOAadPS0dlpZS8HT0KnDx5/fu4u0ubIKKjpRFEN/X//hMRUfuZk4twGEBh/xf+f9h+dDu0nlok/zUZs26eBZdOLkqHJRurth8zlasrEBMjHQ0NwO7dwOefSxsu9u41LnwMSAlZa0lZWzg5AaGh0jq/8eOlDRG2vqmDiIhsGhM7hYX3CcdH932Eif0morNjZ6XDkZ3V24+Zq1MnaSNCWBiQmiqNuO3fD+zaBfz8M3DokHS0N7ELCJDWxw0eLE3xhoQAQ4ZIo31EREQy4VRsK5SuY6d2irYfk1NNjVRDrqpKml6trpamdGtrpSLInToBnTtLh7s74O0N+PhIR1CQNEpIRETUBpyKJZuhbz8WExMDQRCMkjuLtx+Tk7u7dNxwg9KREBERtYiVTcniFG8/RkRE1EFwKrYVnIqVj7mdJ2ymXRkREZGCOBVLNsnR0dHkkiZW71RBRERkBzgVSzZH36ni2qLGFRUViImJQW5urkKRERER2TbVJHZpaWkICwuDq6srPD09TfqMKIpISUlBQEAAunTpgnHjxuHAgQOWDZTaRafT4fHHH292B63+XFJSEnTX9n8lIiIi9SR29fX1iI2Nxdy5c03+zPLly7Fy5UqsWrUKe/bsgZ+fH26//XacP3/egpFSexQVFbXYfgyQkrvy8nIUFRVZMSoiIiJ1UE1il5qaiieeeAJDhw416XpRFJGZmYnFixcjOjoaQ4YMQU5ODi5evIj169dbOFpqK5vsVEFERKQSqknszFVaWooTJ05gwoQJhnPOzs4YO3Ysdu7c2eLn6urqUFNTY3SQ9dh8pwoiIiIbZreJ3YkTJwAAvr6+Rud9fX0N7zUnIyMDHh4ehkOj0Vg0TjIWHh6OwMBAQ/HiawmCAI1Gg/DwcCtHRkREZPsUTexSUlIgCMJ1j+Li4nbd49oEQRTFFpMGAFi4cCGqq6sNR3l5ebvuT+bRd6oAmv7eqapTBRERkQIUrWOXkJCAadOmXfcarVbbpu/28/MDII3cXT1td/LkySajeFdzdnaGMxuzK0rfqaK5OnaZmZmsYycDFn8mIrJPiiZ23t7e8Pb2tsh3BwUFwc/PD/n5+bjlllsASDtrd+zYgWXLllnkniSf6OhoREVF2W3yoWRixeLPRET2SzWdJ8rKynDmzBmUlZVBp9OhpKQEABAcHIyuXbsCAAYOHIiMjAxMmTIFgiAgKSkJ6enp6N+/P/r374/09HS4urpi+vTpCv4kZCpzOlWoiZKJlb7487V1AvXFn9m7l4hI3VTTK3bWrFnIyclpcr6goMDwH39BEJCVlYVZs2YBkNbTpaamYs2aNTh79ixGjRqF119/HUOGDDH5vuwVS3JqKbHSrx+0ZGKl0+mg1WpbrBMoCAICAwNRWlpqNyOjRET2wJxcRDWJnVKY2JFclE6sCgsLERER0ep1V//PEhERKc+cXMRuy50Q2Rqlu2qw+DMRkf1jYkdkJUonViz+TERk/5jYEVmJ0okViz8TEdk/JnZkVTqdDoWFhdiwYQMKCwuh0+mUDslqlE6sWPyZiMj+MbEjq8nNzYVWq0VERASmT5+OiIgIaLVa5ObmKh3adcmVjNpCYqUv/tyrVy+j84GBgSx1QkRkB7grthXcFSsPJct8tIclas41950ajcaqXTXYeYKISD1Y7kRGTOzaT+kyH21lyWSUiRUREZmKiZ2MmNi1nxrrp6k1GSUiIvvDOnZkFaauPduyZYtJ32dL9dOUrjlHRETUFqrpFUu2xdS1ZzqdDu+9955J32lL9dOUrjlHRETUFhyxI7Pp155dO6KlbyR/9S7XoqIinD59utXv9PHxsan6aUrXnCMiImoLJnZkFp1Oh8cff7zJhgIAhnNJSUmGaVlTR7Tuv/9+m1qrpnTNOSIiorZgYkdmMXftmakjWlFRUbLEJxd9zbnmElgW8yUiIlvFxI7MYu7as9ZGvgDY9MhXjx49mpzz8vKy2bp7RETUsTGxI7OYu/astW4LgiDY5MiXfh3hH3/80eS9M2fOKBARERFR65jYkVnasvZMbW2srreOUO/qdYRERES2gokdmaWt/U6jo6Nx/PhxFBQUYP369SgoKEBpaanNJXUAa9gREZF6sY4dmU0/AtdcHbvr9Tt1dHS0mc4S18MadkREpFZM7KhNoqOjERUVZZf9TlnDjoiI1Iq9YlvBXrEdj75PbEVFRYvlTtgnloiIrIW9Yonaoa3rCImIiJTGxI6oGWrbyUtERARwKrZVnIrt2HQ6nV2uIyQiIvUwJxfh5gmi61DLTl4iIiKAiR2RqnFEkYiIrsbEjkilcnNzm60l+Morr3ANIBFRB8XNE0QqpO9le22HjIqKCsTExCA3N1ehyIiISElM7IhU5nq9bEVRhCiKiI+PR319vQLRERGRkpjYEalMa71sAeDUqVPo1asXR+6IiDoYJnZEKmNqj9rTp09zWpaIqINhYkekMub2qE1KSoJOp7NQNEREZEuY2BGpTHh4OAIDA5u0O2uOKIooLy9HUVGRFSIjIiKlMbEjsgKdTofCwkJs2LABhYWF7RpBu7qXralMnb4lIiJ1Y2JHZGG5ubnQarWIiIjA9OnTERERAa1W2661b/petj4+PiZdb+70LRERqRN7xbaCvWKpPfT15q79a6afRv3www/bVUy4vr4evXr1wunTp5t9XxAEBAYGorS0lB0piIhUypxchCN2pFpyTm9aQmv15oD2b2xwcnLCmjVrIAhCkzV3+teZmZlM6oiIOggmdqRKlpjelFtr9ebk2tign5bt1auX0fnAwMB2jwgSEZG6sFcsqU5L05v6dlq2ksyYumFBjo0N0dHRiIqKQlFREaqqquDv74/w8HCO1BERdTBM7EhVWpveFAQBSUlJiIqKUjypMXXDglwbGxwdHTFu3DhZvouIiNSJU7GkKtaa3pRDa/XmBEGARqNBeHi4lSMjIiJ7xcSOVMWa05vtdXW9OW5sICIia2BiR6pi7enN9uLGBiIisibWsWsF69jZFp1OB61Wi4qKimbX2dlq3TadTseNDURE1Cbm5CLcPEGqop/ejImJgSAIRsmdLU9vcmMDERFZA6diSXU4vUlERNQ8TsW2glOxtovTm0RE1BFwKpY6BE5vEhERGeNULBEREZGdYGJHREREZCdUk9ilpaUhLCwMrq6u8PT0NOkzs2bNgiAIRsfo0aMtGygRERGRQlST2NXX1yM2NhZz584163N33HEHqqqqDMenn35qoQiJiIiIlKWazROpqakAgOzsbLM+5+zsDD8/PwtERERERGRbVDNi11aFhYXo2bMnBgwYgDlz5uDkyZNKh0RERERkEaoZsWuLyMhIxMbGok+fPigtLcWSJUswfvx47N27F87Ozs1+pq6uDnV1dYbXNTU11gqXiIiIqF0UHbFLSUlpsrnh2qO4uLjN3z916lTcddddGDJkCCZNmoRt27bh119/xSeffNLiZzIyMuDh4WE4NBpNm+9PREREZE2KjtglJCRg2rRp171Gq9XKdj9/f3/06dMHR44cafGahQsXYv78+YbXNTU1TO6IiIhIFRRN7Ly9veHt7W21+/3xxx8oLy+Hv79/i9c4Ozu3OE1LREREZMtUs8aurKwMZ86cQVlZGXQ6HUpKSgAAwcHB6Nq1KwBg4MCByMjIwJQpU3DhwgWkpKTgnnvugb+/P44fP45FixbB29sbU6ZMMfm++la6XGtHREREStDnIPqc5LpElYiLixMBNDkKCgoM1wAQs7KyRFEUxYsXL4oTJkwQfXx8xM6dO4u9e/cW4+LixLKyMrPuW15e3ux9efDgwYMHDx48rHmUl5e3mrcI/0uIqAWNjY2orKxEt27dIAiCRe6hX8dXXl4Od3d3i9yjo+CzlA+fpXz4LOXDZykfPkv5WPpZiqKI8+fPIyAgAA4O19/3qpqpWKU4ODggMDDQKvdyd3fnXy6Z8FnKh89SPnyW8uGzlA+fpXws+Sw9PDxMus7uCxQTERERdRRM7IiIiIjsBBM7G+Ds7Ixnn32WZVZkwGcpHz5L+fBZyofPUj58lvKxpWfJzRNEREREdoIjdkRERER2gokdERERkZ1gYkdERERkJ5jY2Zi///3v6N27N1xcXODv748HHngAlZWVSoelOsePH8fs2bMRFBSELl26oF+/fnj22WdRX1+vdGiqlJaWhrCwMLi6usLT01PpcFTljTfeQFBQEFxcXDB8+HAUFRUpHZIqff3115g0aRICAgIgCAI2b96sdEiqlJGRgZEjR6Jbt27o2bMnJk+ejF9++UXpsFRp9erVGDZsmKF2XWhoKLZt26Z0WEzsbE1ERAQ2bdqEX375Bf/+979x9OhRxMTEKB2W6hw+fBiNjY1Ys2YNDhw4gJdffhlvvvkmFi1apHRoqlRfX4/Y2FjMnTtX6VBU5f3330dSUhIWL16Mffv2ITw8HJGRkSgrK1M6NNWpra3FTTfdhFWrVikdiqrt2LED8+bNw65du5Cfn4+GhgZMmDABtbW1SoemOoGBgXjhhRdQXFyM4uJijB8/HlFRUThw4ICicXFXrI3bunUrJk+ejLq6OnTu3FnpcFTtxRdfxOrVq3Hs2DGlQ1Gt7OxsJCUl4dy5c0qHogqjRo1CSEgIVq9ebTg3aNAgTJ48GRkZGQpGpm6CICAvLw+TJ09WOhTVO3XqFHr27IkdO3bg1ltvVToc1fPy8sKLL76I2bNnKxYDR+xs2JkzZ7Bu3TqEhYUxqZNBdXU1vLy8lA6DOoj6+nrs3bsXEyZMMDo/YcIE7Ny5U6GoiIxVV1cDAP9tbCedToeNGzeitrYWoaGhisbCxM4GJScnw83NDT169EBZWRm2bNmidEiqd/ToUbz22muIj49XOhTqIE6fPg2dTgdfX1+j876+vjhx4oRCURFdIYoi5s+fjzFjxmDIkCFKh6NKP/30E7p27QpnZ2fEx8cjLy8PgwcPVjQmJnZWkJKSAkEQrnsUFxcbrn/qqaewb98+bN++HY6Ojpg5cyY4Yy4x91kCQGVlJe644w7Exsbi4YcfVihy29OWZ0nmEwTB6LUoik3OESkhISEB+/fvx4YNG5QORbVuuOEGlJSUYNeuXZg7dy7i4uJw8OBBRWPqpOjdO4iEhARMmzbtutdotVrDr729veHt7Y0BAwZg0KBB0Gg02LVrl+LDu7bA3GdZWVmJiIgIhIaG4q233rJwdOpi7rMk83h7e8PR0bHJ6NzJkyebjOIRWdtjjz2GrVu34uuvv0ZgYKDS4aiWk5MTgoODAQAjRozAnj178Morr2DNmjWKxcTEzgr0iVpb6Efq6urq5AxJtcx5lhUVFYiIiMDw4cORlZUFBwcOUF+tPX8uqXVOTk4YPnw48vPzMWXKFMP5/Px8REVFKRgZdWSiKOKxxx5DXl4eCgsLERQUpHRIdkUURcX/e83Ezobs3r0bu3fvxpgxY9C9e3ccO3YMzzzzDPr168fROjNVVlZi3Lhx6N27N1566SWcOnXK8J6fn5+CkalTWVkZzpw5g7KyMuh0OpSUlAAAgoOD0bVrV2WDs2Hz58/HAw88gBEjRhhGjcvKyrjWsw0uXLiA3377zfC6tLQUJSUl8PLyQu/evRWMTF3mzZuH9evXY8uWLejWrZthRNnDwwNdunRRODp1WbRoESIjI6HRaHD+/Hls3LgRhYWF+Oyzz5QNTCSbsX//fjEiIkL08vISnZ2dRa1WK8bHx4v/+c9/lA5NdbKyskQAzR5kvri4uGafZUFBgdKh2bzXX39d7NOnj+jk5CSGhISIO3bsUDokVSooKGj2z2BcXJzSoalKS/8uZmVlKR2a6jz00EOGv9s+Pj7ibbfdJm7fvl3psETWsSMiIiKyE1x0RERERGQnmNgRERER2QkmdkRERER2gokdERERkZ1gYkdERERkJ5jYEREREdkJJnZEREREdoKJHREREZGdYGJHRGQBb731FjQaDRwcHJCZmal0OETUQTCxIyLVmTVrFiZPnmz1+2ZnZ8PT07PV62pqapCQkIDk5GRUVFTgkUceser9iajj6qR0AERE9qasrAyXL1/GXXfdBX9/f6XDadbly5fRuXNnpcMgIplxxI6IVG/cuHFITEzE008/DS8vL/j5+SElJcXoGkEQsHr1akRGRqJLly4ICgrCBx98YHi/sLAQgiDg3LlzhnMlJSUQBAHHjx9HYWEhHnzwQVRXV0MQBAiC0OQegDSqNnToUABA3759DZ8HgI8++gjDhw+Hi4sL+vbti9TUVDQ0NBg+u3LlSgwdOhRubm7QaDR49NFHceHCBUN8Ld1fEARs3rzZKA5PT09kZ2cDAI4fPw5BELBp0yaMGzcOLi4ueO+99wAAWVlZGDRoEFxcXDBw4EC88cYbZj59IrIlTOyIyC7k5OTAzc0N33//PZYvX47nnnsO+fn5RtcsWbIE99xzD3788UfMmDED9913Hw4dOmTS94eFhSEzMxPu7u6oqqpCVVUVFixY0OS6qVOn4osvvgAA7N69G1VVVdBoNPj8888xY8YMJCYm4uDBg1izZg2ys7ORlpZm+KyDgwNeffVV/Pzzz8jJycFXX32Fp59+2qz7X09ycjISExNx6NAhTJw4EW+//TYWL16MtLQ0HDp0COnp6ViyZAlycnLM+l4isiEiEZHKxMXFiVFRUYbXY8eOFceMGWN0zciRI8Xk5GTDawBifHy80TWjRo0S586dK4qiKBYUFIgAxLNnzxre37dvnwhALC0tFUVRFLOyskQPD49W47v2c6IoiuHh4WJ6errRde+++67o7+/f4vds2rRJ7NGjh+F1S/cHIObl5Rmd8/DwELOyskRRFMXS0lIRgJiZmWl0jUajEdevX2907vnnnxdDQ0Ov89MRkS3jGjsisgvDhg0zeu3v74+TJ08anQsNDW3yuqSkxNKhAQD27t2LPXv2GI3Q6XQ6XLp0CRcvXoSrqysKCgqQnp6OgwcPoqamBg0NDbh06RJqa2vh5ubW7hhGjBhh+PWpU6dQXl6O2bNnY86cOYbzDQ0N8PDwaPe9iEgZTOyIyC5cuxFAEAQ0Nja2+jlBEABI06AAIIqi4b3Lly/LFl9jYyNSU1MRHR3d5D0XFxf8/vvvuPPOOxEfH4/nn38eXl5e+OabbzB79uxW4xAEwSjulmK/OjnUP5u3334bo0aNMrrO0dHR5J+LiGwLEzsi6jB27dqFmTNnGr2+5ZZbAAA+Pj4AgKqqKnTv3h0AmozmOTk5QafTteneISEh+OWXXxAcHNzs+8XFxWhoaMCKFSsMSeamTZtMur+Pjw+qqqoMr48cOYKLFy9eNx5fX1/06tULx44dw/3332/uj0NENoqJHRF1GB988AFGjBiBMWPGYN26ddi9ezfWrl0LAAgODoZGo0FKSgqWLl2KI0eOYMWKFUaf12q1uHDhAr788kvcdNNNcHV1haurq0n3fuaZZ3D33XdDo9EgNjYWDg4O2L9/P3766ScsXboU/fr1Q0NDA1577TVMmjQJ3377Ld58802T7j9+/HisWrUKo0ePRmNjI5KTk00qZZKSkoLExES4u7sjMjISdXV1KC4uxtmzZzF//nwTnyoR2RLuiiWiDiM1NRUbN27EsGHDkJOTg3Xr1mHw4MEApKncDRs24PDhw7jpppuwbNkyLF261OjzYWFhiI+Px9SpU+Hj44Ply5ebfO+JEyfi448/Rn5+PkaOHInRo0dj5cqV6NOnDwDg5ptvxsqVK7Fs2TIMGTIE69atQ0ZGhkn3X7FiBTQaDW699VZMnz4dCxYsMCnhfPjhh/HOO+8YSrSMHTsW2dnZCAoKMvnnIiLbIojXLswgIrJDgiAgLy9PkY4VRETWwhE7IiIiIjvBxI6IiIjITnDzBBF1CFx1QkQdAUfsiIiIiOwEEzsiIiIiO8HEjoiIiMhOMLEjIiIishNM7IiIiIjsBBM7IiIiIjvBxI6IiIjITjCxIyIiIrITTOyIiIiI7MT/A4Xd4pFXaAquAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "d=5\n",
    "#和上面展示一致的建模流程\n",
    "LinearR = LinearRegression().fit(X, y)\n",
    "X_ = PF(degree=d).fit_transform(X)\n",
    "LinearR_ = LinearRegression().fit(X_, y)\n",
    "line = np.linspace(-3, 3, 1000, endpoint=False).reshape(-1, 1)\n",
    "line_ = PF(degree=d).fit_transform(line)\n",
    "\n",
    "#放置画布\n",
    "fig, ax1 = plt.subplots(1)\n",
    "\n",
    "#将测试数据带入predict接口，获得模型的拟合效果并进行绘制\n",
    "ax1.plot(line, LinearR.predict(line), linewidth=2, color='green'\n",
    "         ,label=\"linear regression\")\n",
    "ax1.plot(line, LinearR_.predict(line_), linewidth=2, color='red'\n",
    "         ,label=\"Polynomial regression\")\n",
    "\n",
    "#将原数据上的拟合绘制在图像上\n",
    "ax1.plot(X[:, 0], y, 'o', c='k')\n",
    "\n",
    "#其他图形选项\n",
    "ax1.legend(loc=\"best\")\n",
    "ax1.set_ylabel(\"Regression output\")\n",
    "ax1.set_xlabel(\"Input feature\")\n",
    "ax1.set_title(\"Linear Regression ordinary vs poly\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从这里大家可以看出，多项式回归能够较好地拟合非线性数据，还不容易发生过拟合，可以说是保留了线性回归作为线性模型所带的“不容易过拟合”和“计算快速”的性质，同时又实现了优秀地拟合非线性数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，尝试使用多项式回归对鲍鱼数据集进行建模："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:19:15.918084Z",
     "start_time": "2024-03-06T08:19:15.906558Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.linear_model import Ridge, LinearRegression, Lasso\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:21:44.527989Z",
     "start_time": "2024-03-06T08:21:44.515231Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>长度</th>\n",
       "      <th>直径</th>\n",
       "      <th>高度</th>\n",
       "      <th>整体重量</th>\n",
       "      <th>肉重量</th>\n",
       "      <th>内脏重量</th>\n",
       "      <th>壳重</th>\n",
       "      <th>年龄</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.455</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.095</td>\n",
       "      <td>0.5140</td>\n",
       "      <td>0.2245</td>\n",
       "      <td>0.1010</td>\n",
       "      <td>0.150</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.350</td>\n",
       "      <td>0.265</td>\n",
       "      <td>0.090</td>\n",
       "      <td>0.2255</td>\n",
       "      <td>0.0995</td>\n",
       "      <td>0.0485</td>\n",
       "      <td>0.070</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.420</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.6770</td>\n",
       "      <td>0.2565</td>\n",
       "      <td>0.1415</td>\n",
       "      <td>0.210</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.440</td>\n",
       "      <td>0.365</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.5160</td>\n",
       "      <td>0.2155</td>\n",
       "      <td>0.1140</td>\n",
       "      <td>0.155</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0.330</td>\n",
       "      <td>0.255</td>\n",
       "      <td>0.080</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.0895</td>\n",
       "      <td>0.0395</td>\n",
       "      <td>0.055</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   性别     长度     直径     高度    整体重量     肉重量    内脏重量     壳重  年龄\n",
       "0   1  0.455  0.365  0.095  0.5140  0.2245  0.1010  0.150  15\n",
       "1   1  0.350  0.265  0.090  0.2255  0.0995  0.0485  0.070   7\n",
       "2  -1  0.530  0.420  0.135  0.6770  0.2565  0.1415  0.210   9\n",
       "3   1  0.440  0.365  0.125  0.5160  0.2155  0.1140  0.155  10\n",
       "4   0  0.330  0.255  0.080  0.2050  0.0895  0.0395  0.055   7"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入数据集\n",
    "data = pd.read_table('abalone.txt',header=None)\n",
    "data.columns=['性别','长度','直径','高度','整体重量','肉重量','内脏重量','壳重','年龄']\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:21:46.316075Z",
     "start_time": "2024-03-06T08:21:46.297846Z"
    }
   },
   "outputs": [],
   "source": [
    "# 提取特征和标签\n",
    "X = data.iloc[:,:-1]\n",
    "y = data.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:21:51.979500Z",
     "start_time": "2024-03-06T08:21:51.958913Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4177, 8)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:22:21.942219Z",
     "start_time": "2024-03-06T08:22:21.923094Z"
    }
   },
   "outputs": [],
   "source": [
    "poly = PolynomialFeatures(degree=2).fit(X,y) # 学习数据，生成模型\n",
    "X_ = poly.transform(X) # 应用模型完成预测 -> transform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:22:23.207591Z",
     "start_time": "2024-03-06T08:22:23.192401Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4177, 45)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:23:03.492954Z",
     "start_time": "2024-03-06T08:23:03.470443Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['1',\n",
       " '性别',\n",
       " '长度',\n",
       " '直径',\n",
       " '高度',\n",
       " '整体重量',\n",
       " '肉重量',\n",
       " '内脏重量',\n",
       " '壳重',\n",
       " '性别^2',\n",
       " '性别 长度',\n",
       " '性别 直径',\n",
       " '性别 高度',\n",
       " '性别 整体重量',\n",
       " '性别 肉重量',\n",
       " '性别 内脏重量',\n",
       " '性别 壳重',\n",
       " '长度^2',\n",
       " '长度 直径',\n",
       " '长度 高度',\n",
       " '长度 整体重量',\n",
       " '长度 肉重量',\n",
       " '长度 内脏重量',\n",
       " '长度 壳重',\n",
       " '直径^2',\n",
       " '直径 高度',\n",
       " '直径 整体重量',\n",
       " '直径 肉重量',\n",
       " '直径 内脏重量',\n",
       " '直径 壳重',\n",
       " '高度^2',\n",
       " '高度 整体重量',\n",
       " '高度 肉重量',\n",
       " '高度 内脏重量',\n",
       " '高度 壳重',\n",
       " '整体重量^2',\n",
       " '整体重量 肉重量',\n",
       " '整体重量 内脏重量',\n",
       " '整体重量 壳重',\n",
       " '肉重量^2',\n",
       " '肉重量 内脏重量',\n",
       " '肉重量 壳重',\n",
       " '内脏重量^2',\n",
       " '内脏重量 壳重',\n",
       " '壳重^2']"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly.get_feature_names(X.columns) \n",
    "#使用接口get_feature_names来调用生成的新特征矩阵的各个特征上的名称"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:30:17.335461Z",
     "start_time": "2024-03-06T08:30:17.313361Z"
    }
   },
   "outputs": [],
   "source": [
    "reg = LinearRegression().fit(X_,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:30:19.255549Z",
     "start_time": "2024-03-06T08:30:19.242565Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5846827932074914"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.score(X_,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:25:12.839249Z",
     "start_time": "2024-03-06T08:25:12.828449Z"
    }
   },
   "outputs": [],
   "source": [
    "coef = reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:25:24.269076Z",
     "start_time": "2024-03-06T08:25:24.250155Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.84757366e-12, -6.94929391e-02,  3.62086024e+00,  2.57445449e+01,\n",
       "        1.66206992e+01,  3.57200638e+00, -3.42848649e+01,  2.66476767e+01,\n",
       "        3.09594524e+01,  5.92643999e-01,  2.55857378e+00, -4.19172563e+00,\n",
       "        1.80130916e+00,  3.47815448e-01, -5.48518551e-02, -2.06383133e+00,\n",
       "        1.20207226e+00, -4.08529881e+01,  9.39536611e+01, -2.22105174e+01,\n",
       "       -1.42466691e+01,  4.61464990e+01, -7.70215135e+01,  2.35337425e+01,\n",
       "       -1.08227184e+02,  6.08191484e+01,  5.47847470e+01, -4.35158836e+01,\n",
       "       -8.01412933e-01, -8.10680091e+01, -1.08624526e+01,  3.70265753e+01,\n",
       "       -5.82038246e+01, -1.15872253e+02, -7.72034550e+00,  9.29131961e-01,\n",
       "       -4.52102566e+01,  1.26240829e+01,  6.09835112e+00,  5.71699793e+01,\n",
       "        1.97452147e+01,  1.44810160e+01,  2.40047376e+01, -2.37534022e+01,\n",
       "       -7.65932069e+00])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coef"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:25:58.850296Z",
     "start_time": "2024-03-06T08:25:58.827486Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[('1', -1.8475736643474983e-12),\n",
       " ('性别', -0.06949293908753595),\n",
       " ('长度', 3.6208602359363207),\n",
       " ('直径', 25.744544897917066),\n",
       " ('高度', 16.620699158015185),\n",
       " ('整体重量', 3.5720063772000334),\n",
       " ('肉重量', -34.28486486244363),\n",
       " ('内脏重量', 26.6476766572343),\n",
       " ('壳重', 30.95945235281875),\n",
       " ('性别^2', 0.5926439993225876),\n",
       " ('性别 长度', 2.5585737754714017),\n",
       " ('性别 直径', -4.191725633616849),\n",
       " ('性别 高度', 1.8013091575425872),\n",
       " ('性别 整体重量', 0.34781544768902806),\n",
       " ('性别 肉重量', -0.05485185511889687),\n",
       " ('性别 内脏重量', -2.063831333746669),\n",
       " ('性别 壳重', 1.2020722645520117),\n",
       " ('长度^2', -40.852988146886894),\n",
       " ('长度 直径', 93.9536611464311),\n",
       " ('长度 高度', -22.210517399580485),\n",
       " ('长度 整体重量', -14.246669067638644),\n",
       " ('长度 肉重量', 46.14649904306213),\n",
       " ('长度 内脏重量', -77.0215135299217),\n",
       " ('长度 壳重', 23.533742451290337),\n",
       " ('直径^2', -108.22718410350582),\n",
       " ('直径 高度', 60.819148409376254),\n",
       " ('直径 整体重量', 54.78474697304248),\n",
       " ('直径 肉重量', -43.51588359030037),\n",
       " ('直径 内脏重量', -0.8014129329954712),\n",
       " ('直径 壳重', -81.06800908342517),\n",
       " ('高度^2', -10.862452554286584),\n",
       " ('高度 整体重量', 37.02657526732674),\n",
       " ('高度 肉重量', -58.20382457951651),\n",
       " ('高度 内脏重量', -115.8722528740604),\n",
       " ('高度 壳重', -7.720345501300712),\n",
       " ('整体重量^2', 0.9291319606108208),\n",
       " ('整体重量 肉重量', -45.210256628459355),\n",
       " ('整体重量 内脏重量', 12.624082852472633),\n",
       " ('整体重量 壳重', 6.098351124546933),\n",
       " ('肉重量^2', 57.16997927044254),\n",
       " ('肉重量 内脏重量', 19.74521470956966),\n",
       " ('肉重量 壳重', 14.48101604540249),\n",
       " ('内脏重量^2', 24.004737583603543),\n",
       " ('内脏重量 壳重', -23.753402174152615),\n",
       " ('壳重^2', -7.659320686858382)]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[*zip(poly.get_feature_names(X.columns),reg.coef_)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:28:04.509070Z",
     "start_time": "2024-03-06T08:28:04.476362Z"
    },
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>-0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>性别</td>\n",
       "      <td>-0.069493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>长度</td>\n",
       "      <td>3.62086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>直径</td>\n",
       "      <td>25.744545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>高度</td>\n",
       "      <td>16.620699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>整体重量</td>\n",
       "      <td>3.572006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>肉重量</td>\n",
       "      <td>-34.284865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>内脏重量</td>\n",
       "      <td>26.647677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>壳重</td>\n",
       "      <td>30.959452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>性别^2</td>\n",
       "      <td>0.592644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>性别 长度</td>\n",
       "      <td>2.558574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>性别 直径</td>\n",
       "      <td>-4.191726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>性别 高度</td>\n",
       "      <td>1.801309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>性别 整体重量</td>\n",
       "      <td>0.347815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>性别 肉重量</td>\n",
       "      <td>-0.054852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>性别 内脏重量</td>\n",
       "      <td>-2.063831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>性别 壳重</td>\n",
       "      <td>1.202072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>长度^2</td>\n",
       "      <td>-40.852988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>长度 直径</td>\n",
       "      <td>93.953661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>长度 高度</td>\n",
       "      <td>-22.210517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>长度 整体重量</td>\n",
       "      <td>-14.246669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>长度 肉重量</td>\n",
       "      <td>46.146499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>长度 内脏重量</td>\n",
       "      <td>-77.021514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>长度 壳重</td>\n",
       "      <td>23.533742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>直径^2</td>\n",
       "      <td>-108.227184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>直径 高度</td>\n",
       "      <td>60.819148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>直径 整体重量</td>\n",
       "      <td>54.784747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>直径 肉重量</td>\n",
       "      <td>-43.515884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>直径 内脏重量</td>\n",
       "      <td>-0.801413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>直径 壳重</td>\n",
       "      <td>-81.068009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>高度^2</td>\n",
       "      <td>-10.862453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>高度 整体重量</td>\n",
       "      <td>37.026575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>高度 肉重量</td>\n",
       "      <td>-58.203825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>高度 内脏重量</td>\n",
       "      <td>-115.872253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>高度 壳重</td>\n",
       "      <td>-7.720346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>整体重量^2</td>\n",
       "      <td>0.929132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>整体重量 肉重量</td>\n",
       "      <td>-45.210257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>整体重量 内脏重量</td>\n",
       "      <td>12.624083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>整体重量 壳重</td>\n",
       "      <td>6.098351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>肉重量^2</td>\n",
       "      <td>57.169979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>肉重量 内脏重量</td>\n",
       "      <td>19.745215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>肉重量 壳重</td>\n",
       "      <td>14.481016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>内脏重量^2</td>\n",
       "      <td>24.004738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>内脏重量 壳重</td>\n",
       "      <td>-23.753402</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>壳重^2</td>\n",
       "      <td>-7.659321</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            0           1\n",
       "0           1        -0.0\n",
       "1          性别   -0.069493\n",
       "2          长度     3.62086\n",
       "3          直径   25.744545\n",
       "4          高度   16.620699\n",
       "5        整体重量    3.572006\n",
       "6         肉重量  -34.284865\n",
       "7        内脏重量   26.647677\n",
       "8          壳重   30.959452\n",
       "9        性别^2    0.592644\n",
       "10      性别 长度    2.558574\n",
       "11      性别 直径   -4.191726\n",
       "12      性别 高度    1.801309\n",
       "13    性别 整体重量    0.347815\n",
       "14     性别 肉重量   -0.054852\n",
       "15    性别 内脏重量   -2.063831\n",
       "16      性别 壳重    1.202072\n",
       "17       长度^2  -40.852988\n",
       "18      长度 直径   93.953661\n",
       "19      长度 高度  -22.210517\n",
       "20    长度 整体重量  -14.246669\n",
       "21     长度 肉重量   46.146499\n",
       "22    长度 内脏重量  -77.021514\n",
       "23      长度 壳重   23.533742\n",
       "24       直径^2 -108.227184\n",
       "25      直径 高度   60.819148\n",
       "26    直径 整体重量   54.784747\n",
       "27     直径 肉重量  -43.515884\n",
       "28    直径 内脏重量   -0.801413\n",
       "29      直径 壳重  -81.068009\n",
       "30       高度^2  -10.862453\n",
       "31    高度 整体重量   37.026575\n",
       "32     高度 肉重量  -58.203825\n",
       "33    高度 内脏重量 -115.872253\n",
       "34      高度 壳重   -7.720346\n",
       "35     整体重量^2    0.929132\n",
       "36   整体重量 肉重量  -45.210257\n",
       "37  整体重量 内脏重量   12.624083\n",
       "38    整体重量 壳重    6.098351\n",
       "39      肉重量^2   57.169979\n",
       "40   肉重量 内脏重量   19.745215\n",
       "41     肉重量 壳重   14.481016\n",
       "42     内脏重量^2   24.004738\n",
       "43    内脏重量 壳重  -23.753402\n",
       "44       壳重^2   -7.659321"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([poly.get_feature_names(X.columns),reg.coef_.tolist()]).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:28:18.141388Z",
     "start_time": "2024-03-06T08:28:18.117079Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/baoyiyang/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>高度 内脏重量</td>\n",
       "      <td>-115.872253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>直径^2</td>\n",
       "      <td>-108.227184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>直径 壳重</td>\n",
       "      <td>-81.068009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>长度 内脏重量</td>\n",
       "      <td>-77.021514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>高度 肉重量</td>\n",
       "      <td>-58.203825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>整体重量 肉重量</td>\n",
       "      <td>-45.210257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>直径 肉重量</td>\n",
       "      <td>-43.515884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>长度^2</td>\n",
       "      <td>-40.852988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>肉重量</td>\n",
       "      <td>-34.284865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>内脏重量 壳重</td>\n",
       "      <td>-23.753402</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>长度 高度</td>\n",
       "      <td>-22.210517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>长度 整体重量</td>\n",
       "      <td>-14.246669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>高度^2</td>\n",
       "      <td>-10.862453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>高度 壳重</td>\n",
       "      <td>-7.720346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>壳重^2</td>\n",
       "      <td>-7.659321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>性别 直径</td>\n",
       "      <td>-4.191726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>性别 内脏重量</td>\n",
       "      <td>-2.063831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>直径 内脏重量</td>\n",
       "      <td>-0.801413</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>性别</td>\n",
       "      <td>-0.069493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>性别 肉重量</td>\n",
       "      <td>-0.054852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>-0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>性别 整体重量</td>\n",
       "      <td>0.347815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>性别^2</td>\n",
       "      <td>0.592644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>整体重量^2</td>\n",
       "      <td>0.929132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>性别 壳重</td>\n",
       "      <td>1.202072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>性别 高度</td>\n",
       "      <td>1.801309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>性别 长度</td>\n",
       "      <td>2.558574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>整体重量</td>\n",
       "      <td>3.572006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>长度</td>\n",
       "      <td>3.62086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>整体重量 壳重</td>\n",
       "      <td>6.098351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>整体重量 内脏重量</td>\n",
       "      <td>12.624083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>肉重量 壳重</td>\n",
       "      <td>14.481016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>高度</td>\n",
       "      <td>16.620699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>肉重量 内脏重量</td>\n",
       "      <td>19.745215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>长度 壳重</td>\n",
       "      <td>23.533742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>内脏重量^2</td>\n",
       "      <td>24.004738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>直径</td>\n",
       "      <td>25.744545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>内脏重量</td>\n",
       "      <td>26.647677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>壳重</td>\n",
       "      <td>30.959452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>高度 整体重量</td>\n",
       "      <td>37.026575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>长度 肉重量</td>\n",
       "      <td>46.146499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>直径 整体重量</td>\n",
       "      <td>54.784747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>肉重量^2</td>\n",
       "      <td>57.169979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>直径 高度</td>\n",
       "      <td>60.819148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>长度 直径</td>\n",
       "      <td>93.953661</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      feature        coef\n",
       "33    高度 内脏重量 -115.872253\n",
       "24       直径^2 -108.227184\n",
       "29      直径 壳重  -81.068009\n",
       "22    长度 内脏重量  -77.021514\n",
       "32     高度 肉重量  -58.203825\n",
       "36   整体重量 肉重量  -45.210257\n",
       "27     直径 肉重量  -43.515884\n",
       "17       长度^2  -40.852988\n",
       "6         肉重量  -34.284865\n",
       "43    内脏重量 壳重  -23.753402\n",
       "19      长度 高度  -22.210517\n",
       "20    长度 整体重量  -14.246669\n",
       "30       高度^2  -10.862453\n",
       "34      高度 壳重   -7.720346\n",
       "44       壳重^2   -7.659321\n",
       "11      性别 直径   -4.191726\n",
       "15    性别 内脏重量   -2.063831\n",
       "28    直径 内脏重量   -0.801413\n",
       "1          性别   -0.069493\n",
       "14     性别 肉重量   -0.054852\n",
       "0           1        -0.0\n",
       "13    性别 整体重量    0.347815\n",
       "9        性别^2    0.592644\n",
       "35     整体重量^2    0.929132\n",
       "16      性别 壳重    1.202072\n",
       "12      性别 高度    1.801309\n",
       "10      性别 长度    2.558574\n",
       "5        整体重量    3.572006\n",
       "2          长度     3.62086\n",
       "38    整体重量 壳重    6.098351\n",
       "37  整体重量 内脏重量   12.624083\n",
       "41     肉重量 壳重   14.481016\n",
       "4          高度   16.620699\n",
       "40   肉重量 内脏重量   19.745215\n",
       "23      长度 壳重   23.533742\n",
       "42     内脏重量^2   24.004738\n",
       "3          直径   25.744545\n",
       "7        内脏重量   26.647677\n",
       "8          壳重   30.959452\n",
       "31    高度 整体重量   37.026575\n",
       "21     长度 肉重量   46.146499\n",
       "26    直径 整体重量   54.784747\n",
       "39      肉重量^2   57.169979\n",
       "25      直径 高度   60.819148\n",
       "18      长度 直径   93.953661"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#放到dataframe中进行排序\n",
    "coeff = pd.DataFrame([poly.get_feature_names(X.columns),reg.coef_.tolist()]).T\n",
    "coeff.columns = [\"feature\",\"coef\"]\n",
    "coeff.sort_values(by=\"coef\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现，不仅数据的可解释性还存在，我们还可以通过这样的手段做特征工程——特征创造。多项式帮助我们进行了一系列特征之间相乘的组合，若能够找出组合起来后对标签贡献巨大的特征，那我们就是创造了新的有效特征，对于任何学科而言发现新特征都是非常有价值的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在鲍鱼数据集上来再次确认多项式回归提升模型表现的能力："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:29:40.295030Z",
     "start_time": "2024-03-06T08:29:40.259498Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5278909357356787"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg = LinearRegression().fit(X,y)\n",
    "reg.score(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:30:43.552365Z",
     "start_time": "2024-03-06T08:30:43.531780Z"
    }
   },
   "outputs": [],
   "source": [
    "#查看一下多项式变化之后，模型的拟合效果如何了\n",
    "poly = PolynomialFeatures(degree=3,interaction_only=True).fit(X,y)\n",
    "X_ = poly.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:31:00.839633Z",
     "start_time": "2024-03-06T08:31:00.739776Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2:0.589889546103672\n",
      "time:0.09390687942504883\n"
     ]
    }
   ],
   "source": [
    "from time import time\n",
    "time0 = time()\n",
    "reg_ = LinearRegression().fit(X_,y)\n",
    "print(\"R2:{}\".format(reg_.score(X_,y)))\n",
    "print(\"time:{}\".format(time()-time0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 切分训练集和测试集\n",
    "Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = 3  #注意如果设置过大，可能报错内存不足\n",
    "poly = PolynomialFeatures(degree=d,interaction_only=True).fit(Xtrain,Ytrain)\n",
    "Xtrain_ = poly.transform(Xtrain)\n",
    "Xtest_ = poly.transform(Xtest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5984152149848969"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg = LinearRegression(fit_intercept=False).fit(Xtrain_,Ytrain)\n",
    "reg.score(Xtrain_,Ytrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5305000793115451"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.score(Xtest_,Ytest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:32:29.834541Z",
     "start_time": "2024-03-06T08:32:29.381769Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2:0.7894980494621255\n",
      "time:0.018660783767700195\n"
     ]
    }
   ],
   "source": [
    "# 尝试换其他模型？\n",
    "from sklearn.tree import DecisionTreeRegressor as DTR\n",
    "\n",
    "time0 = time()\n",
    "print(\"R2:{}\".format(DTR(max_depth=10).fit(X,y).score(X,y)))\n",
    "print(\"time:{}\".format(time()-time0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-03-06T08:33:14.121518Z",
     "start_time": "2024-03-06T08:33:12.250947Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2:0.9358558684413865\n",
      "time:1.411072015762329\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor as RFR\n",
    "\n",
    "time0 = time()\n",
    "print(\"R2:{}\".format(RFR(n_estimators=100).fit(X,y).score(X,y)))\n",
    "print(\"time:{}\".format(time()-time0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "到这里，多项式回归就全部讲解完毕了。多项式回归主要是通过对自变量上的次方进行调整，来为线性回归赋予更多的学习能力，它的核心表现在提升模型在现有数据集上的表现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本节大家学习了多元线性回归，岭回归，Lasso和多项式回归总计四个算法，他们都是围绕着原始的线性回归进行的拓展和改进。其中岭回归和Lasso是为了解决多元线性回归中使用最小二乘法的各种限制，主要用途是消除多重共线性带来的影响并且做特征选择，而多项式回归解决了线性回归无法拟合非线性数据的明显缺点，核心作用是提升模型的表现。"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.11"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
